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SUMMARY 


BACKGROUND 

This  review  of  team  .raining  and  evaluation  was  undertaken  by  H RRO  in 
order  to  provide  an  information  base  that  could  be  used  by  DARPA  as  a foundation 
to  facilitate  decisions  regarding  future  research  program  support.  in  order 
to  pursue  their  mission  of  operational  readiness,  the  Services  conduct  most  of 
their  training  in  the  operational  commands.  However,  in  the  past,  most  training 
research  has  focused  on  individual  training  at  schools  and  at  other  institutional 
locations.  Considering  the  amount  of  team  training  that  goes  on  in  the  Services, 
eitner  formally  recognized  as  training  or  combined  with  operations,  the  funds 
committed  to  R&D  support  are  relatively  small.  There  is  a need,  therefore,  for 
Increased  research  emphasis  on  team  training  and  evaluation. 

PURPOSE 

The  purpose  of  this  literature  review  was  to  provide  information  to  DARPA 
which  would  be  useful  for  planning  research  and  development  programs  in  the  area 
of  team  training.  Such  programs  are  needed  to  develop  improved  team  training 
methods  and  technologies,  as  well  as  Lr  provide  new  measurement  techniques  and 
procedures  for  evaluation.  The  specific  objectives  in  preparing  this  review 
were  to  address  the  following  questions: 

• What  state-of-art  gaps  are  there  in  team  training  strategies  and 
evaluation  techniques? 

• What  new  team  training  strategies  appear  to  hold  promise  for 
application  to  the  DoD  environment? 

e What  new  evaluation  techniques  can  be  used  to  assess  team  t ain  v ? 


i 


1 


' ...  - 


METHOD 


A thorough  search  was  made  of  the  major  documentation  sources  for  public  a 
tions  which  were  relevant  to  team  training/evaluation . Initial  sources  searched 
included  the  Educational  Resources  Information  Center  (ERIC),  National  Technical 
Information  Service  (NTIS),  and  Defense  Documentation  Center  (DDC)  indices.  In 
addition  to  DoD-sponsored  research  reports,  publications  from  the  social 
psychological  areas  such  as  group  dynamics  and  the  industrial  training  field 
were  surveyed.  Secondary  sources  and  cross-references  were  then  identified  and 
reviewed.  HumRRO  personnel  with  experience  in  the  team  training  area  were 
consulted  for  information  about  current  military  team  training  methods.  The  most 
recent  military  doctrine  regarding  team  training  procedures  was  obtained  and 
r*  'ewed.  Military  personnel  were  contacted  in  all  the  Services  to  identify  team 
training  approaches  now  in  operation.  Current  team  training  and  evaluation 
studies  underway  within  the  Army  Research  Institute  and  the  other  Services  were 
examined  and  described. 

FINDINGS  AND  IMPLICATIONS 

A classification  scheme  was  introduced  in  which  the  training  techniques  and 
situations  discussed  in  this  review  were  categorized  along  two  dimensions.  On 
one  dimension,  a distinction  was  made  between  'team  training  and  multi-individual 
training.  Multi-individual  training  occurs  in  a group  context  but  focuses  on  the 
development  of  individual  skills.  On  the  other  hand,  the  focus  of  team  training 
is  on  developing  team  skills  (e.g.,  coordination,  cooperation,  etc.).  The  type 
of  task  situation  was  the  other  dimension  used  to  classify  the  training  techniques 
reviewed  in  this  study.  Task  situations  were  categorized  as  either  "established” 
or  "emergent."  Established  situations  are  those  in  which  the  tasxs  and  the 
activities  required  to  perform  these  tasks  can  be  almost  completely  specified. 
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Emergent  situations  are  those  in  which  all  tasks  and  activities  cannot  be  specified 
and  the  probable  consequences  of  certain  actions  cannot  be  predicted.  This  type 
of  situation  allows  for  unanticipated  behaviors  to  emerge. 

Team  training  studies  and  practices  were  categorized  according  to  the 
classification  scheme  described  above.  These  studies  were  described  in  this 
review  as  having  followed  two  conceptual  models  of  team  behavior — Stimulus- 
Response  (S-R)  and  Organismic.  The  S-F.  model  adherents  tended  to  study  team 
training  in  laboratory  settings  derived  from  established  task  situations.  More 
realistic  environments  were  used  by  other  researchers  who  attended  to  emergent 
factors  in  the  job  situation  (the  organismic  approach).  It  was  this  latter 
group  of  investigators  who  demonstrated  the  need  for  training  in  team  skills,  e ren 
though  individual  skill  proficiency  was  found  to  be  a prerequisite  for  effective 
team  training  and  performance.  Other  conclusions  which  were  drawn  from  the 
literature  are: 

• The  team  context  is  not  the  proper  location  for  initial  individual 

♦ skill  acquisition. 

• Performance  feedback  is  critical  to  the  learning  of  team  as  well  as 
individual  skills. 

Current  military  team  training  practices  were  described  as  varying  in  cost, 
in  degree  of  fidelity  to  the  operational  site,  and  in  the  variety  and  complexity 
of  the  tasks  trained.  Subjective  methods  cf  evaluation  were  found  to  be  pre- 
dominant . 

Evaluation  of  team  training  effectiveness  depends  upon  the  development  of 
objective  team  performance  measurement  instruments  and  procedures.  This  review 
of  the  literature  and  current  practice  support  the  conclusion  that  this  area 
contains  many  problems  which  must  be  solved  before  substantial  improvement  in 
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team  training  evaluation  can  occur.  The  production  of  standardizable , replicable 
test  conditions  and  the  establishment  of  accepted  performance  criteria  are  two 
such  problems. 

A phased  research  program  was  suggested  as  warranting  support  in  order  to 
overcome  gaps  in  t a current  state  of  knowledge.  The  major  factor  in  this  program 
is  the  need  for  methods  which  differentiate  individual,  multi -individual,  and  team 
skills  and  training  requirements.  Such  techniques,  which  could  be  integrated  with 
job/task  analyses,  would  discriminate  between  isolated  and  Interactive  behaviors. 
Possible  techniques  to  study  for  this  purpose  are  variations  of  interaction  analys 
and  other  unobtrusive  measures.  Successful  development  of  this  methodology  will 
provide:  an  opportunity  to  establish  team  performance  standards  criteria  as  part 
of  training  requirements  analyses.  Development  of  these  techniques  would  then 
permit  other  research  studies  to  be  accomplished  effectively. 

Issues  identified  in  this  review  which  merit  research  support  include  the 
following : 

• Team  Feedback 

What  changes  in  training  methods  or  content  result  from  feedback? 

Which  method  of  providing  team  feedback  is  most  useful  (e.g.,  video 
recording  versus  post-exercise  debriefing)? 

• Assessment  Training 

Will  training  cf  team  members  to  assess  their  own  performance  result  in 
greater  team  proficiency  than  if  no  self-assessment  training  was  provided? 
Will  training  to  assess  the  performance  of  other  team  members  affect 
proficiency? 

• Simulation  Fidelity 

What  degree  of  simulation  fidelity  is  critical  for  effective  team 
training?  Can  two-sided  engagement  simulation  techniques  be  applied  to 
the  training  of  team  skills?  Although  low  fidelity  simulations  were 
found  to  be  ineffective  in  established  task  situations,  would  they  be 
more  useful  in  emergent  situations? 
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• Team  Composition/Structure 

What  are  the  effects  of  degree  of  team  member  familiarity  on  the 
acquisition  of  team  skills?  Will  the  length  of  time  that  a team  functions 
together  effect  proficiency? 

• Skill  Training  Sequencing 

Is  it  more  1 fficient  to  train  individual  skills  first  and  then  team 
skills,  or  team  skills  first,  or  both  concurrently? 

Combined  studies  are  feasible  which  deal  with  more  than  one  of  the  above 
research  issues.  For  example,  feedback  or  skill  sequencing  issues  can 
be  studied  in  environments  which  vary  in  their  degree  of  simulation  fidelity, 


or  in  task  situations  which  are  established  or  emergent. 

The  studies  that  were  suggested  above  point  out  the  considerable  research 
and  development  effort  needed  in  the  team  training/evaluation  area.  With  further 
support,  our  state  of  knowledge  in  the  area  should  expand  considerably. 
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PREFACE 

This  report  contains  a critical  review  of  the  literature  relevant 
to  team  training.  Current  instructional  and  evaluative  techniques  within  the 


Services  were  surveyed  and  described.  State-of-art  gaps  were  identified  and 
research  needs  dociu  ented. 

This  study  was  performed  in  HumRRO's  Eastern  Division,  Dr.  J.  Daniel  Lyons, 
Director.  The  woi't  was  performed  under  the  general  direction  and  supervision 
of  Dr.  Robert  J.  Seidel,  Program  Director,  Instructional  Technology  Group.  The 
authors  of  this  report  gratefully  acknowledge  Dt . Seidel's  assistance  in  the 
identification  of  potential  team  training  research  areas. 

This  reseat  ch  was  supported  by  the  Defense  Advanced  Research  Projects  Agency 
under  DARPA  Order  No.:  3187;  Contract  No.:  MDA903  76  C 0210.  The  Technical 
Monitor  for  this  project  was  Dr.  Harold  F.  O’Neil,  Jr.  His  support  and  substan- 
tive assistance  in  this  study  is  gratefully  appreciated.  The  authors  also  wish 
to  express  their  gratitude  for  the  bibliography  provided  by  Dr.  John  Germas  of 
the  Army  Research  Institute. 
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Interdependent,  coordinated  team  performance  is  a predominant  character- 
istic of  most  operational  activities  within  the  Service*-, . Training  of  teams  in 
operational  units  is  the  transition  between  initial  training  and  combat.  The 
influence  of  effective  team  behavior  upon  syecem  performance  underlines  its 
consequences  for  training.  This  training  is  more  difficult  and  assumed  to  be 
more  costly  to  do  than  is  individual  training.  It  is  usually  performed  in 
operational  environments  or  in  high  fidelity  simulations  cf  them.  For  these 
reasons,  there  is  a nerd  for..aeund  data  to  determine  the  relative  effectiveness 
of  alternate  training  concepts  as  a means  for  improving  team  training. 

BACKGROUND 

In  order  to  pursue  their  mission  of  operational  readiness,  the  Services 
conduct  most  (up  to  90%)  of  their  training  in  the  operational  commands.  However, 
in  the  past,  most  training  research  has  focused  on  individual  training  at 
Schools  and  at  other  institutional  locations. 

MG  Gorman,  Deputy  Chief  of  Staff  for  Training,  TRADOC,  recently  stated  [94] 
that  there  is  a need  for  increased  research  emphasis  on  what  is  called  collec- 
tive training"  (team  or  crew  tactical  training  in  a unit  context) . A recent 
Defense  Science  Board  task  force  report  [74]  stated  that  in  view  of  the  amount 
of  team  training  that  goes  on  in  the  Services,  either  formally  recognized  as 
training  or  combined  with  operations,  the  funds  committed  to  R&D  support  for 
team  training  are  relatively  small.  In  fiscal  year  1974,  the  Army  Researct 
Institute  (ARI)  spent  less  than  $2  million  on  research  in  unit  training.  Although 
it  was  the  largest  such  program  of  research  in  history,  it  amounted  to  only  11% 


9 


....... 


a.-  tt  1.  . k ii  r . _ 


« fr 

fl 

U 


of  the  human  resources  research  program  in  that  year.  Most  of  the  money  went 

t 

for  improved  institutional  training  or  exploration  of  other  individual  training 
issues . 

This  review  of  the  team  training  area  was  undertaken  by  HumFRO  in  order 
to  provide  an  information  base  that  could  be  used  by  ARPA  and  the  Services  as  a 
foundation  tc  facilitate  decisions  regarding  future  research  program  support. 

PURPOSE 

The  purpose  of  this  review  was  to  provide  information  to  DARPA  which  would 
be  useful  for  planning  research  and  development  programs  in  the  area  of  team 
training.  Such  programs  are  -na^ded  to  develop  improved  team  training  methods 
and  technologies  and  to  provide  new  measurement  techniques  and  procedures  for 
evaluation.  The  following  objectives  were  established  to  guide  the  preparation 
of  this  review: 

1.  Describe  existing  instructional  strategies  and  evaluation  techniques 
relevant  to  team  training. 

2.  Identify  state-of-art  gaps  in  team  training  and  evaluation  and 
suggest  new  team  training  strategies  and  evaluation  techniques  which 
warrant  further  study. 

APPROACH 

Review  of  Technical  Literature 

An  extensive  search  was  made  of  the  major  documentation  sources  in  order 
to  amass  an  initial  listing  of  publications  which  were  relevant  to  team  training. 
The  emphasis  was  on  the  applicability  of  the  document  to  tactical  training  of 
teams.  However,  instructional  strategies  and  evaluation  techniques  which  had 
been  used  in  other  contexts  were  examined  for  their  possible  relevancy.  The 
following  major  sources  were  searched: 
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1.  Educational  Resources  Information  Center  (ERIC).  These  indices  were 

rearched  for  relevant  titles  over  the  preceding  10-year  period  .he  descriptors 
used  in  this  search  included:  team  training,  military  training,  industrial 

training,  group  dynamics,  group  effectiveness,  group  instruction,  small  group 
instruction,  training  techniques. 

2.  Defense  Documentation  Center  (DDC) . The  indices  of  the  preceding  10- 
year  period  were  searched  using  the  following  descriptors:  team  training,  group 
dynamics,  group  problem  solving,  training  crews. 

3.  Psychological  Abstracts.  The  descriptors  used  to  search  these  indices 

were:  group  dynamics,  training,  training/industrial,  training/military,  and 

group/small.  Here  too,  a 10-year  period  was  selected  for  the  in-depth  search. 

4.  National  Technical  Information  Service  (NTIS).  The  general  area  of 
"military  sciences"  was  canvassed  for  appropriate  literature.  Also,  team 
training,  group  dynamics,  group  problem  solving  and  group  training  were  used  as 
descriptors . 

5.  HumRRO  Library.  This  library  specializes  in  publications  which  are 
relevant  to  military  training  and  evaluation.  A sizeable  number  of  documents 
were  thus  obtained  "in-house"  for  this  review. 

Following  the  development  of  an  initial  list  which  contained  many  hundreds 
of  titles,  a brief  review  of  each  document's  abstract  was  performed  to  insure 
that  the  review  would  focus  only  on  those  publications  which  were  relevant  to 
team  training  and  evaluation.  Relevant  documents  were  then  obtained  and 
reviewed. 

Once  the  search  in  the  initial  sources  was  completed,  secondary  sources 
(i.e.,  Journal  of  Applied.  Psychology,  Training  in  Business  and  Industry ) were 
canvassed  for  relevant  articles.  In  addition,  "key"  documents  that  were  more 
than  10  years  old  were  identified  from  the  cross-references  in  documents 
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already  reviewed.  These  "key"  documents  were  also  obtained  and  reviewed.  AU 
these  items  are  listed  in  the  References  Section  of  this  report. 

Sample  of  Current  Military  Team  Training  Techniques 

The  most  recent  military  doctiine  regarding  team  training  procedures  was 
obtained  and  reviewed.  Current  studies  underway  within  the  Army  Research 
Institute  and  the  other  Services  were  examined.  Military  personnel  were  contacted 
in  all  the  Services  to  identify  team  training  approaches  currently  in  operation. 
HumRRQ  personnel  with  experience  in  the  team  training  area  (e.g.,  Dr.  Joseph 
Olmstead)  were  consulted  for  information  about  current  military  team  training 
methods.  Examples  of  current  techniques  were  then  chosen  for  inclusion  in  this 
report  °nd  are  described  in  Chapter  IV. 

DEFINITIONS 

As  can  be  seen  from  the  variety  of  sources  chosen  for  the  initial  literature 
search  described  above,  a cleat  conceptual  delineation  of  the  team  training  area 
had  not  been  made  at  the  time  by  HumRRO  reviewers.  Team  training  research  was 
viewed  as  part  of  the  more  general  subject  matter  of  small  group  research  or 
group  dynamics.  This  larger  field  had  generated  literally  thousands  of  empirical 
studies  and  theoretical  papers,  most  of  which  were  only  tangentially  relevant 
to  the  sub-area  of  team  training.  An  understanding  was  needed  of  what  makes  the 
team  context  unique. 

Before  a state-of-art  review  on  team  training  could  be  accomplished,  one 
needed  to  define  what  constitutes  a "team."  In  size,  teams  may  be  two  or  more 
individuals  with  no  upper  limitdef ined . Teams  also  may  be  characterized  by 
virtually  any  degree  of  formal  organizational  structure  and  permanency  [41 j.1 

1This  document  by  Hall  and  Rizzo  provides  an  excellent  description  of  how  defini- 
tional problems  have  troubled  the  team  training  field  and  the  various  ways  the 
research  literature  has  dealt  with  these  difficulties. 
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Rather  than  attempting  one  more  definition,  the  authors  viewed  th*- 

L 

description  givep  by  Glaser,  Klaus  and  Egerman  [33]  as  providing  the  most 
meaningful  distinction  of  small  groups  and  teams  considered  in  this  review. 

4 *• 

Teams,  have  the  following  characteristics: 

f 

1.  They  are  relatively  rigid  in  structure,  organization,  and 
communication  pattern. 

2.  The  task  of  each  team  member  is  well  defined. 

3.  The  functioning  of  the  team  depends  upon  the  coordinated  parti- 
cipation of  all  or  several  individuals. 

-or 

In  contrast,  small  groups'  differ  in  that  they  generally: 

1 

1.  Have  an  indefinite  or  loose  structure,  organization,  and  communica- 
tion pattern. 

2.  Have  assignments  which  are  assumed  in  the  course  of  group  inter- 
action rather  than  designated  beforehand. 

3.  The  group  product  can  be  a function  of  one  or  more  of  the  group 
members  involved  depending  upon  the  quality  and  quantity  of  their 
participation." 

These  definitions  were  used  to  guide  the  final  selection  of  studies  and  papers 
for  in-depth  review. 

Instructional  strategies  and  evaluation  techniques  '..inch  are  relevant  to 
"teams"  should  take  into  consideration  the  dimensions  described  above.  In 
addition,  teams  are  goal-  or  mission-oriented  [41  ] and  so  the  specific  context 
in  which  the  team  will  operate  must  be  considered  before  any  training  or 
evaluation  technique  can  be  applied. 


0 
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The  question  of  whether  "team"  or  "multi-individual”  behaviors  are  involved 
in  a given  context  has  not  been  adequately  studied.  As  derived  from  the  above 

discussion,  team  training  can  be  defined  as: 

• The  training  of  two  or  more  individuals  who  are  associated 
together  in  work  or  activity.  The  team  is  relatively  rigid 
in  structure  and  communication  pattern.  It  Is  goal-  or 
mission-oriented  with  the  task  of  each  team  member,  well- 
defined.  The  functioning  of  the  team  depends  upon  the 
coordinated  participation  of  all  or  several  individuals. 

The  focus  of  team  training  and  feedback  is  on  team  skills 
(e.g.,  coordination),  activities,  and  products. 

On  the  other  hand,  multi-individual  training  can  be  defined  as: 

• The  training  of  two  or  more  individuals  who  are  associated  in 
a group  context.  The  trainees  may  or  may  not  be  part  of  a 
team.  However,  the  focus  of  the  training  and  feedback  is  on 
individual  skills,  activities,  and  products. 

Methods  are  needed  by  which  one  can  analyze  given  task  situations  and  derive 

individual  and  team  training  requirements. 

In  the  chapters  that  follow,  a distinction  will  be  made  on  this  dimension  of 
whether  the  study  or  technique  reviewed  is  pertinent  to  multi-individual  or 


team  contexts. 


Chapter  II:  REVIEW  OF  TEAM  TRAINING  RESEARCH 

Research  in  the  area  of  team  training  has  been  going  on  for  approximately 
20  years,  albeit  at  a relatively  low  level  of  effort  [21,  22,  37,  55,  62,  64,  70, 
71].  Glanzer  and  Glaser  [35]  stated  tha.  despite  the  importance  of  the  area, 
relatively  little  formal  knowledge  exist  i concerning  methods  of  describing  and 
analyzing  team  performance.  The  complexity  of  team  training  problems  was  cited 
as  contributing  to  that  situation— "In  the  investigation  of  the  areas  of  team 
as  opposed  to  individual  training  and  performance,  problems  appear  of  an  entirely 
new  order  of  magnitude"  [35]. 

In  1975  the  Defense  Science  Board  described  several  difficulties  in  perform- 
ing team  training  research  which  have  prevented  significant  breakthroughs: 

This  kind  of  R&D  must  be  piggybacked  on  operations  in  the  field, 
large  numbers  of  R&D  personnel  are  required,  the  opportunities  for 
data  collection  during  the  exercise  are  marginal,  inferential 
statistics  and  psychometrics  were  not  designed  for  this  order  of 
complexity,  there  are  limited  opportunities  for  repeated  trials,  the 
ultimate  test  of  team  training  is  combat,  which  cannot  be  simulated.  L./'U 

The  difficulties  cited  in  doing  team  training  research  have  prevented  a 
thorough  study  of  the  problems  and  research  needs  identified  during  the  inter- 
vening years.  However,  there  has  been  some  definite  progress  and  certain 
research  trends  identified  and  described  [361 

Research  findings  on  some  major  team  problems  and  issues  will 

be  described  in  the  eight  sections  that  fo  he  first  section  describes 

the  "Stimulus-Response"  and  "Organismic"  models  which  underlie  different  approaches 
to  the  study  of  team  training  [1].  Next  (Section  2),  the  components  of  emergent 
versus  established  situations  and  their  effects  on  team  training  research  are 
described.  A discussion  of  the  relative  importance  and  interactions  of 
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individual  training  and  team  training  (Section  3)  is  followed  by  descriptions 
of  and  problems  involved  in  studying  team  skills  (Section  4).  In  Section  5 
the  degree  of  simulation  fidelity  in  conjunction  wit.i  its  influence  on  effective 
training  is  examined.  This  is  followed  by  a description  (Section  6)  of  the 
effects  of  feedback  on  team  performance  and  techniques  used  to  present  knowledge 
of  results  (KOR)  to  team  members.  Section  7 examines  team  structure  and 
composition  and  their  effects  on  team  training.  Finally,  in  Section  8 the 
benefits  of  a systems  approach  to  the  development  of  effective  team  training  is 
discussed. 

1.  TWO  CONCEPTUAL  APPROACHES  TO  TEAM  TRAINING  RESEARCH 

Alexander  and  Cooperband  [1]  have  distinguished  between  two  team  training 
models  based  on  the  situations  in  which  team  behavior  takes  place.  The 
variables  and  environment  selected  for  study  by  researchers  depend  upon  the  team 
model  chosen.  The  "Stimulus-Response"  model  is  applied  to  teams  which  operate 
primarily  in  established  situations,  whose  tasks,  and  the  activities  required  to 
perform  these  tasks,  can  be  almost  completely  specified,  and  the  assignment  of 
functions  among  team  members  and  their  equipment  is  relatively  rigid.  This 
viewpoint  underlies  attempts  to  apply  certain  principles  and  techniques  of 
individual  learning  (e.g.,  operant  conditioning)  to  research  on  team  training. 

Studies  performed  at  the  American  Institutes  for  Research  Team  Training  Laboratory 
are  illustrative  of  the  model  [53].  For  example,  in  one  study  reinforcement  of 
team  response  was  manipulated  to  measure  differences  in  team  member  proficiency. 
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In  the  "Organismic"  model  [1]  the  team  is  considered  to  be  a synthetic 
organism  of  which  individuals  are  components.  This  model  is  oriented  toward 
teams  operating  in  environments  which  include  a significant  proportion  of 
emergent  situations.  Although  there  are  defined  task  assignments,  the  individual 
has  a considerable  degree  of  discretion  in  how  to  perform  the  task  under  various 
contingencies.  Consequently,  proficient  team  performance  can  depend  upon  the 
development  of  appropriate  team  procedures  for  coping  with  the  environment  by 
its  members.  Thus,  adaptive  innovations  are  required  and  decision-making  and 
problem-solving  3kills  are  emphasized  [1].  An  example  of  this  is  the  work  don* 
by  Chapman,  et  al  [97].  Teams  in  a simulated  air  defense  direction  center  were 
presented  with  sudden  disruptive  changes  in  their  environment  and  were  required 
to  react  to  the  new  stimuli  quickly  in  order  to  solve  given  problems  precisely. 

2.  ESTABLISHED  VS.  EMERGENT  SITUATIONS 

The  two  conceptual  approaches  described  in  Section  1 take  as  their  basis 
the  situational  context  in  which  team  behavior  occurs.  This  context  is 
actually  a continuum  of  situations,  the  end  points  of  which  are  described  as 
established  or  emergent . Boguslaw  and  Porter  [10]  define  these  situations  as 
follows: 

An  established  situation  is  one  in  which  (1)  all  action-relevant 
environmental  conditions  are  specifiable  and  predictable,  (2)  all 
action-relevant  states  of  the  system  are  specifiable  and  predictable, 
and  (3)  available  research  technology  or  records  are  adequate  to 
provide  statements  about  the  probable  consequences  of  alternative 
actions.  An  emergent  situation  is  one  in  which  (1)  all  action- 
relevant environmental  conditions  have  not  been  specified,  (2)  the 
state  of the  system  does  not  correspond  to  relied-upon  predictions, 

(3)  analytic  solutions  are  not  available,  given  the  current  state  of 
analytic  technology  [10]. 

An  established  situation  was  the  primary  focus  of  a research  program  of 
the  Team  Training  Laboratory  of  the  American  Institute  for  Research  [54] . 

The  advantages  of  such  a situation  for  research  are: 
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• it  is  relatively  easy  to  abstract  key  characteristics  of  real- 
world  team  tasks  and  represent  them  in  the  laboratory; 

• experiments  can  be  done  relatively  inexpensively; 

• studies  can  be  conducted  in  reasonable  time  periods;  and 

• monitoring  of  team  activities  can  be  feasibly  accomplished. 

On  the  other  hand,  simplifying  the  representation  of  team  functions  can  mask 
or  omit  possible  important  variables  which  influence  behavior  in  the  real  world. 
By  abstracting  the  situational  context  in  the  laboratory,  there  can  be  a loss 
of  opportunity  for  trainees  to  react  to  breakdowns  and  problems  which  may  arise 
in  an  operational  setting. 

Providing  skills  to  deal  with  emergent , unstructured  situations  was  seen 
as  a major  goal  of  team  training  in  studies  by  the  System  Development  Corporation 
(SDC)  [1].  The  development  of  coordinative  skills  was  stressed  although  it  was 
recognized  that  these  are  based  upon  a certain  minimum  attainment  of  individual 
skill.  Team  training  devices  and  techniques  were  seen  as  requiring  orientation 
toward  the  training  of  innovative  behaviors  and  skills  necessary  to  adapt  to 
unforeseen  problems  [11,  23]. 

An  emergent  situation  permits  a more  realistic,  less  abstract,  approach  to 
the  investigation  of  team  training  variables  than  does  an  established  situation. 
When  a team  was  formed  in  the  laboratory,  its  prior  motivational  characteristics 
were  important  determinants  of  performance.  But,  when  teams  were  studied  while 
carrying  out  face  valid  activities,  prior  motivational  states  appeared  to  be 
less  important  [29].  What  was  important  in  this  case  was  training  in  coordina- 
tion such  that  team  members  became  fully  aware  of  their  responsibilities  to 
compensate  for  the  inabilities  of  others,  or  to  overcome  temporary  problems 
when  the  situation  called  for  it. 
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EMERGENT  ESTABLISHED 


By  viewing  task  situations  as  established  or  emergent,  one  can  classify 
the  training  techniques  and  research  studies  discussed  in  this  review  as  pertinent 
to  either  type  of  context.  Similarly,  the  distinction  between  team  and  multi- 
individual training  requirements  discussed  earlier  comprises  a second  classifies— 
tory  dimension.  The  resulting  2x2  matrix  constitutes  a classification  scheme 
which  will  be  used  in  the  remaining  sections  of  this  chapter  and  in  later  chapters 
to  categorize  the  research  studies  and  training  procedures  surveyed.1 

3.  INDIVIDUAL  VS.  TEAM  TRAINING 

The  studies  in  this  section  were  classified  as  shown  in  Table  1.  Although 
the  focus  of  this  literature  review  was  on  iojxm  training,  there  were  some  studies 
surveyed  that  dealt  with  individual  skills  training  in  a multi-individual  context. 


TRAINING  FOCUS 

team  MULTI-INDIVIDUAL 


Klaus  & Glaser  [51,52,54] 
Hall  & Rizzo  [41] 

Briggs  & Johnston  [11,14] 

Horrocks  [43,44] 

Kanarick,  et  ^1  [49] 
Johnston  [47] 

McRae  [59] 

Williges  [87] 

Alexander  & Cooperband  [1] 

Kanarick,  e£  al  [49] 

Table  1.  Classification  of  studies  reviewed  in  Chapter  II,  Section  3, 
"Individual  vs.  Team  Training." 


The  classification  of  studies  throughout  this  document  represent  the  judgments 
of  the  authors.  No  formal  rating  system  was  used. 
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One  common  conclusion  0f  the  research  on  team  training,  regardless  of 
whether  it  was  based  on  an  "Organismic"  or  "Stimulus-Response"  viewpoint,  is 
that  individual  proficiency  is  the  basis  for  an  effective  team  [49].  Inus, 
in  emergent  situations,  developing  team  awareness  or  abilities  to  deal  with 
unexpected  problems  requires  that  each  team  member  has  attained  the  requisite 
job  skills.  Team  training  is  likely  to  proceed  most  efficiently  when  the  team 
members  have  thoroughly  mastered  their  own  specific  assignments  beforehand 
[54].  Horrocks,  et  al  [43,  44]  found  that  when  team  coordination  was  emphasized 
early  in  training,  it  interfered  with  acquisition  of  individual  task  competence. 

The  question  could  then  be  asked  if  team  training  was  of  any  benefit  at 
all  and,  if  so,  to  what  degree  and  in  what  manner.  Is  it  possible  to  predict 
when  individual  training  will  be  superior  to  team  training?  [14]  Briggs  and 
Johnston  [11]  concluded  their  review  finding  no  direct  evidence  for  the  super- 
iority of  team  training  over  individual  training.  They  interpreted  this  find- 
ing as  indicating  a need  for  team  training  devices  to  include  provisions  for  either 
refreshing  or  augmenting  individual  skills.  In  other  laboratory  investigations 
(derived  from  established  situations)  which  compared  individual  with  team 
training,  equivocal  results  were  produced.  Horrocks,  et  al  [44]  found  that 
if  a member  of  an  intact  team  was  replaced  by  another  equally  competent  person, 
there  was  no  detriment  in  team  performance.  This  implied  that  there  did  not 
appear  to  be  a generalised  "team  skill"  which  operated  independently  of  specific 
task  competencies  [11].  Evidence  supporting  this  finding  came  from  other  AIR 
laboratory  studies  [51,  52].  In  these  studies  only  the  proficiency  level  of 
the  members  at  the  start  of  team  training  determined  team  performance.  As  these 
task  situations  were  routine  and  well-established,  team  performance  was  seen 
to  be  the  sum  total  of  individual  performances  [41]. 
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On  the  other  hand,  Johnston  [47]  studied  two-person  teams  working  in  a 
simulated  radar  situation  where  the  objective  was  to  track  and  intercept 
incoming  aircraft.  The  amount  of  coordination  and  communication  in  the  eraining 
task  was  varied  and  demonstrated  that  team  training  was  better  than  individual 
training  when  the  criterion  task  required  coordination  between  individuals. 

In  such  emergent  situations,  training  in  "teem  skills"  (e.g..  coordination) 
was  required  and  effective  performance  was  regarded  as  something  more  than  the 

sum  of  individual  skills  [59]. 

Verbal  communication  training  was  found  not  to  be  useful  when  the  task 
did  not  require  the  skill  [47].  In  a study  by  Williges,  et  al  [89]  it  was 
concluded  that  verbal  communication  played  an  insignificant  role  in  team  work 
and  this  role  was  not  enhanced  by  verbal  training.  In  this  study,  two-person 
teams  were  required  to  coordinate  interception  of  two  planes  on  a simulated 
radar  screen.  In  one  condition  team  members  received  only  verbal  communication 
concerning  the  performance  ofthe  other  team  member;  in  another  condition,  the 
team  members  were  given  verbal  communication  and  visual  access  to  the  radar  screen 
of  the  other  member.  It  was  found  that  verbal  communication  facilitated 
performance  of  the  team  only  when  there  was  no  other  more  efficient  communication 
channel  available.  In  the  conditions  where  verbal  and  visual  communication  both 

were  allowed,  verbal  communication  had  no  effect  on  team  performance. 

' > 

Alexander  and  Cooperband  [1]  proposed  that  team  training  should  be 
effective  when:  the  training  stresses  the  acquisition  of  coordinative  skills, 

and  the  tasks  being  trained  are  such  that  formal  rules  cannot  be  stated  and 
procedures  must  be  developed  by  the  team  in  the  process  of  task  accomplishment. 
Kanarick,  et  a]L  [49]  proposed  three  phases  for  team  training. 
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Initially,  there  is  the  need  to  train  individuals  in  the  procedural 
aspects  of  their  jobs,  doctrine,  and  the  process  approach  to  decision 
making.  This  training  should  be  followed  by  a phase  in  which  team 
members  are  instructed  as  a unit,  learning  the  interactive  and 
communicative  requirements  of  team  functioning.  The  final  phase  is 
devoted  to  tactical  training  where  teams  are  taught  to  apply  their 
procedural  and  interactive  skills  to  certain  situations  requiring  inno- 
vative and  creative  behaviors. 


4.  TEAM  SKILLS 

Studies;  in  this  Section  are  classified  in  Table  2. 


TRAINING  FOCUS 


TEAM 


p 

a 

CO 

H 

t 

CO 

w 


< 

ZD 


CO 


^ e> 
ia<3 

4/1  H 
< fa 


MULTI -INDIVIDUAL 


\ 

\ 

1 

Sidorsky  & Simoneau  [80] 

\ 

Sidorsky  & Houseman  [81] 
Alexander  & Cooperband  [1 ] 

i 

Federman  & Siegel  [27]  j 

i 

r , i 

Boguslaw  & Porter  [10]  j 

Kanarick,  ej:  al_  [49] 

i 

Hausser,  et  a]L  [42] 

Table  2.  Classification  of  studies  reviewed  in  Chapter  II,  Section  4, 
"Team  Skills."  ! 


As  can  be  seen,  training  in  team  Skills  is  important  for  application  to  emergent 

i 

situations  [80,81].  What  are  these  skills  and  how  should  they  be  trained? 
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With  respect  to  one  such  skill — cooperation — Alexander  and  Cooperband  [1]  have 

r' 

described  it, as  follows: 

Learning  to  cooperate  means  learning  the  strengths  and  weaknesses  of 
one  another,  learning  when  the  others  want  help  snd  when  they  do  not 
want  it,  learning  to  pace  one's  activities  to  fit  the  needs  of  all, 
and  learning  to  behave  so  that  one's  actions  are  not  ambiguous. 

Nonetheless,  one  of  the  problems  in  the  area  of  "team  skills"  is  a lack  of 

unambiguous,  operational  definitions  of  skills  such  as  coordination,  interaction, 

cooperation,  etc.  [27].  As  thgxf  aj~e  conflicting  vii  regarding  these  terms, 

so  too  are  there  problems  in  training  and  evaluation  of  these  skills  [91]. 

One  of  the  most  difficult  skills  to  train,  yet  one  of  the  most  important 

in  dealing  with  team  functions,  is  skill  in  tH^Bhalysis  of  one's  own  errors. 

Boguslaw  and  Porter  [10]  reported  that  it  did  not  appear  possible  to  specify 

how  to  train  this  skill.  It  was  hypothesized  that  it  would  help  to  present 

trainees  with  numerous  unusual  situations  which  could  not  be  handled  by  simple 

adherence  to  standard  operating  procedures.  Problem-solving  discussions  (rather 

than  didactic  presentations)  ulso  could  have  helped.  An  effective  team  performer 

must  learn  when  he/she  is  reaching  the  point  of  being  overloaded  and  needs  assistance. 

Studies  are  needed  to  examine  the  kinds  of  "self-evaluative"  skills  which  can 

enhance  team  performance  an!  to  identify  techniques  which  could  be  used  to  train 

these  skills. 

The  training  of  team  awareness,  a skill  considered  of  importance  to  performance 
In  emergent  situations  [49],  can  benefit  from  the  application  of  a number  of 
existing  technologies.  The  split-  or  multiple-screen  technique  of  television  can 
be  used  to  show  team  members  their  jobs  in  relation  to  other  team  members. 

Using  this  technique,  a trainee  could  observe  how  a given  action  serves  as  a 
stimulus  for  some  other  team  member.  Such  training  provides  the  trainees  with  a 
better  appreciation  of  their  individual  roles  in  the  team  and  of  the  need  for 
effective  communication  and  interaction. 
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A study  was  conducted  to  investigate  the  application  of  the  PLATO  IV  Computer 
Assisted  Instruction  (CAI)  system  to  training  interpersonal  skills  [42].  Th^ 
content  areas  chosen  included!  feedback,  communication,  goal  setcing,  problem 
solving,  decision  making,  and  reinforcement.  Training  had  a positive  effect  on 
some  of  the  skill  performances  of  company  commanders  in  leading  recruit  companies 
through  a nine-week  training  period.  However,  in  most  of  the  behavioral  measures 
of  company  commander  performance,  CAI  training  did  not  have  a discernible  effect. 
Further  investigation  of  the  usefulness  of  CAI  as  an  approach  to  training  team 
skills  appears  needed. 


5.  SIMULATION  FIDELITY 

Studies  discussed  in  this  Section  are  classified  as  shown  in  Table  3. 
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Briggs  & Johnston  [11-14] 

Bachrach  [3] 

Alexander  & Cooperband  [1] 

Madden  [60] 

Prophet  & Caro  [73] 

Table  3.  Classification  of  studies  reviewed  in  Chapter  II,  Section  5, 
"Simulation  Fidelity." 


In  training  teams  or  studying  their  behavior  in  emergent  situations,  high 
fidelity  simulations  may  be  contemplated  [60,  73].  High-fidelity  simulation 
can  be  both  costly  and  time  consuming;  space,  equipment,  and  staffing  constitute 
substantial  investments  in  money  and  time.  The  problems  of  measurement  are 


24 


great  because  of  th*'  interactions  of  the  many  variables  involved.  Once  a large- 
scale  simulation  exercise  has  begun,  the  experimenters  have  little  control  over 
the  events  which  follow  [3]. 

High-fidelity  simulation  of  complex  team  tasks  are  exemplified  by  those 
which  simulated  air  traffic  control  tasks  and  air  defense  operations  which 
utilized  teams  ranging  from  30-40  people.  These  tasks  approached  very  closely 
the  complexities  of  the  operational  task,  and  task  fidelity  was  very  high  since  in 
some  cases  an  operational  system  itself  was  used  in  training  [1]. 

Can  low-fidelity  simulations  be  used  adequately  for  team  training?  A 
series  of  experiments  by  Briggs,  et^  al  [11,  12,  13,  14]  have  led  to  the  conclusion 
that  team  training  did  not  transfer  well  to  the  operational  environment  when  the 
training  simulation  was  of  low  fidelity.  However,  . ese  studies  were  all  performed 
in  contexts  derived  from  established  situations.  No  studies  were  found  using  low- 
fidelity  simulations  in  emergent  situations.  Recent  techniques  used  in  the  Army 
to  simulate  two-sided  comba':  engagements  have  employed  low-cost,  high-fidelity 
procedures  in  emergent  contexts.  These  techniques  will  be  discussed  further  in 
later  chapters  of  this  report. 

6.  FEEDBACK/ KNOWLEDGE  OF  RESULTS 

Studies  on  performance  feedback  discussed  in  this  Section  were  classified 
as  shown  in  Table  4.  Most  of  this  research  was  performed  in  laboratory 
settings. 
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Egerman,  et  al  [25,26] 
Glaser,  et^  al  [33] 

Klaue  & Glaser  [51,52,54] 
Short,  et  al  [77] 

Glanzer  [34] 

Cockrell  [18] 

Cockrell  & Sadacca  [19] 
Burnstein  & Wolff  [15] 
Klaus,  j2t  a_l  [53] 

Marra  [61 ] 

Wolff,  e_t  al  [90,92] 

Nebeker,  £t^  ^1  [65] 
Burnstein  & Wolff [15] 
Wolff,  et  al  [90,92] 

Kanarick  [49] 

Alexander  & Cooperband  [1] 

Table  4.  Classification  of  studies  reviewed  in  Chapter  II,  Section  6, 
"Feedback/Knowledge  of  Results." 

Performance  feedba  f.  Ls  unquestionably  the  single  most  critical  parameter 
in  team  or  individual  training,"  concluded  Kanarick,  et  al  [49]  in  a review  of 
the  research  literature  on  feedback.  The  series  of  studies  by  AIR  [25,  26,  33, 
51,  52,  54,  77]  considered  an  important  characteristic  of  teams  to  be  the  fact 


that  regardless  of  how  a correct  team  response  was  made,  every  member  of  the 


team  was  expwsed  to  the  same  reinforcing  event.  This  was  true  even  when  some 
team  members  made  errors  or  were  idle.  These  studies  demonstrated  that  team 
reinforcement  might  also  foster  unwanted,  inappropriate  responses  that  could 
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create  a significant  decrement  in  team  performance.  The  results  of  this  program 
of  research  suggested  several  guidelines  which  should  be  employed  for  team 
training. 

. . . it  appears  essential  that  team  practice  result  in  clear 
and  immediate  reinforcement  following  each  correct  team  response; 
practice  in  the  absence  of  team  reinforcement  for  criterion-level 
performance  is  more  than  likely  to  lead  to  a decrement  in  team 
proficiency  [54]. 

An  approach  used  to  provide  team  members  with  feedback  or  Knowledge  of 
Results  is  the  post-exercise  debriefing.  Debriefing  sessions  following  team  train- 
ing sessions  are  an  opportunity  for  the  team  to  examine  individual  proficiencies 
and  to  explore  alternative  ways  of  organizing  the  task  so  as  to  develop  more 
efficient  and  proficient  team  performance.  However,  the  post-exercise  discussion 
as  feedback  may  have  an  undesirable  characteristic — a large  time  gap  between  the 
response  and  the  feedback  [34].  The  amount  of  elapsed  time  before  feedback  is 
given  should  be  studied  systematically  to  determine  the  effects  of  delayed 
feedback. 

In  a study  by  Nebeker,  et_  al  [65],  the  results  showed  that  although 
individuals  performed  better  with  feedback  than  without,  it  did  not  matter 
whether  the  feedback  was  raw  or  percentile  scores  or  whether  it  concerned  the 
individual,  his/her  group,  or  both.  The  results  underscored  the  importance  of 

feedback  as  a general  factor  which  augmented  and  sustained  performance.  Other 
studies  of  feedback  have  demonstrated  that  reinforcement  contingencies  affect 
team  performance,  and  have  .llustrated  the  effectiveness  of  feedback  or  Knowledge 
of  Results  to  sustain  performance  [15,  53,  61,  90,  92]. 

Team  members  receive  feedback  information  from  intrinsic  and/or  extrinsic 
sources.  Extrinsic  feedback  is  provided  by  a source  external  to  the  system,  such 
as  an  instructor.  Intrinsic  feedback  is  received  while  the  team  is  performing  its 
tasks  and  is  inherent  in  the  task  itself.  The  effectiveness  of  intrinsic  feedback 
has  been  shown  in  a variety  of  studies  [18,  19]. 
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Providing  feedback  in  team  training  presents  problems  beyond  those 
encountered  in  training  individuals. 

These  problems  arise  from  three  considerations.  (1)  There 
may  be  several  criteria  of  effective  team  performance  with  no  clear- 
cut  tradeoffs  among  them;  these  criteria  may  be  vague  and  difficult 
to  state  objectively  and  may  change  during  system  operations.  (2) 

In  order  for  a team  to  operate  effectively,  it  is  necessary  for  its 
members  to  develop  and  maintain  individual  skills  as  well  as  skill  in 
working  together;  there  is  a possibility  that  these  skills  may 
require  different  feedback  procedures  which  may  mutually  interfere. 

(3)  When  a complex  system  operates,  there  is  usually  a large  volume 
of  information  available  about  the  state  of  the  environment,  the 
state  of  the  system,  and  the  performance  of  system  personnel;  some 
of  this  information  may  be  conducive  and  some  inimical  to  effective 
learning  Cl]. 

A major  source  of  difficulty  in  team  training  is  the  identification  and  correc- 
tion of  individual  errors.  In  many  complex  team  tasks,  there  are  often  many 
possible  correct  procedures,  making  error  identification  difficult.  These 
conditions  give  rise  to  the  problem  of  monitoring  or  supervision  of  team 
responses  to  evaluate  their  adequacy. 

Although  the  studies  cited  above  discuss  the  general  effectiveness  of 
feedback,  more  work  needs  to  be  done  to  systematically  investigate  the  methods 
which  might  be  used  to  deliver  feedback  information.  Various  techniques  such 
as  videorecording  and  post-training  debriefings  are  now  used  to  inform  team 
members  about  their  responses.  These  and  other  methods  need  to  be  studied 
regarding  their  effects  on  team  performance. 


7.  TEAM  STRUCTURE  AND  COMPOSITION 

Studies  discussed  in  this  Section  were  classified  as  shown  in  Table  5. 
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Briggs  & Johnston  [H»13] 

Bolin  [9] 

Klaus  [54] 

Nebeker  [65]  1 

Bolin  [9] 

Egerman,  et  al^  [25,26] 

Johnston  & Briggs  [48] 

George  [30] 
Moore  [63] 

i Baldwin,  e_t  al  [6] 

Table  5.  Classification  of  studies  reviewed  in  Chapter  II,  Section  7, 
"Team  Structure  and  Composition." 


Although  not  directly  related  to  team  training  techniques,  questions 
regarding  *-he  effects  of  group  structure  and  composition  upon  training  are 
important  for  understanding  the  effects  of  training  upon  team  performance  [6]. 

Two  types  of  team  structure  were  identified  and  studied  in  a series  of  laboratory 
experiments  [11].  In  a serial  or  vertical  structure  activities  are  sequential 
with  input  from  one  team  member  based  on  output  of  another.  Parallel  team 
structures  (e.g.,  rowing  team)  do  rot  have  the  same  member  interactions. 

Briggs  and  Johnston  [13]  suggested  that  parallel  team  structures  are  pre- 
ferable to  serial  structures.  This  was  because  team  performance  in  the  parallel 
structure  was  not  dependent  on  the  least  skilled  member.  However,  Klaus  [54] 
reported  that  the  parallel  structure  led  to  only  a short-term  gain  and  eventually 
to  a decrement  in  team  performance. 
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A variety  of  studies  on  team  structure,  workload  allocation,  physical 
arrangement,  and  member  replacement  yielded  conflicting  results  [38,  39].  For 
example,  Bolin,  e_t  a^l  [9]  found  in  studying  task  size  and  team  performance  that 
fully  cooperative  three-man  teams  performed  better  than  individuals  working  alone 
on  image  interpretation.  Moore  [63]  and  George  [30]  studied  the  effects  of  task 
characteristics  on  team  performance  in  emergent  settings.  Moore  found  that 
increasing  the  task  load  on  teams  did  not  affect  team  performance  when  all  the 
information  necessary  for  task  completion  was  furnished  to  the  team.  George 
showed  teams  performed  better  when  they  were  required  to  use  a communication 
network  which  included  all  of  the  team  members.  In  a study  conducted  in  an 
established  task  context  to  test  the  effects  of  team  arrangement  on  performance 
of  radar  controller  teams  [48],  investigators  found  that  team  communication  inhibited 
team  performance  in  some  cases.  In  several  studies  occurring  in  established 
task  situations,  team  member  replacement  was  found  to  be  relatively  unimportant 
and  dependent  on  the  skill  levels  of  the  persons  involved  [24,  25,  26,  39].  Yet, 
personnel  turnover  was  considered  to  be  a major  factor  degrading  team  performance  in 
an  operational  setting.  For  example,  a tank  commander  reported  that  with  40% 
turnover  in  crews  every  90  days,  one  cannot  expect  acceptable  levels  of  team 
proficiency  [74].  There  is  a need  to  systematically  investigate  the  effects  on 
team  proficiency  of  the  length  of  time  a team  performs  together  under  established 
or  emergent  conditions. 

With  such  conflicting  findings,  the  search  for  superior  team  training 
methods  has  been  slow.  Task-  or  situation-specific  characteristics  may  account 
for  many  of  the  conflicting  findings  obtained  in  team  training  research.  A 
more  systematic  approach  to  the  research  area  appears  necessary.  This  suggestion 
is  amplified  in  the  section  below. 
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8.  SYSTEMS  APPROACH  TO  TEAM  TRAINING 


The  most  common  framework  for  research  in  the  team  training  field  has  been 
to  study  team  behavior  in  the  same  manner  in  which  individual  behavior  was 
investigated.  The  team  was  viewed  by  the  "Stimulus-Response"  adherents  as  a 
"single  response  unit"  [54].  The  team  product  as  a whole  was  the  focus  of 
research  rather  than  its  component  parts.  This  made  it  feasible  to  investigate 
the  effects  of  various  conditions  of  feedback  upon  team  proficiency.  The 
fact  that  the  team  response  might  have  been  quite  complex  and  result  from  the 
integration  of  separate  responses  of  several  team  members  was  of  interest  only 
to  investigators  who  followed  the  "Organ ismic"  viewpoint. 

Both  sets  of  researchers  favor  the  application  of  a "systems  approach"  or 
instructional  system  development  (JSD)  model  to  the  design  and  development  of 
team  training  exercises,  materials,  methods,  and  devices.  Implementing  such  an 
approach  could  yield  information  necessary  for  the  identification  of  critical 
variables  to  be  studied  as  part  of  an  extensive  program  of  team  training  research. 

Teams  are  created  or  defined  to  accomplish  certain  goals  or  missions  [28]. 

The  relationships  among  team  members  are  meaningful  only  to  the  extent  that  they 
are  involved  in  attainment  of  the  goals.  Thus,  before  any  team  training  is 
undertaken,  an  in-depth  analysis  of  the  system  siiouLd  be  performed  [72].  The 
task  characteristics  need  to  be  identified  and  training  objectives  derived  [40,  46]. 
The  members  of  the  team  should  be  made  aware  of  the  team's  goals  and  objectives. 
Exercises  and  instructional  materials  should  be  structured  with  respect  to  the 
objectives,  and  the  training  situation  should  be  made  as  similar  as  possible  to 
the  operational  criterion  situation  [8,  34].  Finally,  the  quality  control 
components  of  the  system  should  provide  feedback  data  to  trainees  and  trainers 
for  the  purpose  of  improving  the  training  s vs  ton  and  team  proficiency. 
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The  major  difficulties  to  overcome  in  order  to  implement  an  ISD  approach  to 
team  training  are  its  expense,  the  need  for  qualified  personnel,  and  obtaining 
the  resources  and  authority  to  carry  out  the  training  research  and  development 
deemed  necessary. 

* .1 
: l 

SUMMARY 

Two  approaches  to  the  study  of  team  training  were  described.  These 
research  trends  were  based  upon  what  was  termed  the  Stimulus— Response  (S  R) 
and  "Organismic"  models.  The  former  model  led  to  laboratory  investigations 
which  addressed  team  training  problems  occurring  in  established  task  situations. 
Adherents  of  the  Organismic  model  tended  to  study  team  training  in  more  realistic 
surroundings,  as  they  were  more  interested  in  the  emergent  phenomena  present  in 
the  task  situation.  The  studies  performed  by  both  group  of  investigators 
cited  above  suggested  the  following  conclusions: 

• Team  training  is  a necessary  addition  to  individual  training  for 
tasks  which  require  interaction  and  other  "team  skills." 

• Effective  team  training  can  only  occur  if  the  team  members  enter 

the  training  situation  with  the.  requisite  individual  skill  competencies. 

« The  team  context  is  not  the  appropriate  location  for  initial 
individual  skill  acquisition. 

• Performance  feedback  is  critical  to  team  as  well  a^  individual 
skill  acquisition. 

The  research  findings  discussed  a *ove  which  show  that  a particular  team 
structure  is  "better,"  or  that  one  type  of  reinforcement  schedule  is  more 
effective  than  another,  are  quite  fragile  results.  They  can  be  largely  obviated 
by  such  things  as  lack  of  fidelity  in  the  training  situation  [16].  The  nature 
of  the  training  situation  is  a critical  element  to  be  concerned  with  for  future 
team  training  research.  Up  to  now  the  tasks  used  to  study  team  training,  with 
some  notable  exceptions,  have  been  quite  limited. 
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Recent  programs  in  the  Army  have  demonstrated  the  possibility  of  developing 
high  fidelity,  yet  low-cost  simulations  which  may  provide  team  training 
researchers  with  the  controls  they  need  to  study  "team  skills"  in  realistic 
environments  rather  than  in  the  laboratory  [74].  Other  implications  for  research 
generated  by  the  review  above  will  be  discussed  further  in  Chapter  V.  Finally 

the  research  literature  has  shown  that  weaknesses  exist  in  conceptual  defini- 
tions of  such  terms  as  "team,"  "coordination,"  "fidelity,"  "interaction,"  etc. 
Efforts  are  needed  to  provide  a more  precise  taxonomic  underpinning  to  this  field 
to  serve  as  a basis  for  more  generalizeable  and  comparable  research. 
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Chapter  III:  REVIEW  OF  EVALUATION  TECHNIQUES  FOR  TEAM  TRAINING 


The  importance  of  methods  for  systematically  evaluating  team  training 
effectiveness  and  the  difficulty  in  developing  such  methods  has  been  widely 
acknowledged  for  a number  of  years  [32,  33,  34,  67,  74];  Glanzer  and  Glaser  in 
1955  [35]  noted  that  there  was  an  absence  of  satisfactory  proficiency  measures 
for  team  performance.  In  1962,  Glaser  [31]  remarked: 

With  respect  to  proficiency  measurement,  less  work  has  been  done 
in  this  area  than  has  been  carried  out  with  individual  performance, 
yet  the  importance  of  developing  proficiency  measures  for  multiman 
systems  is  being  more  frequently  considered. 

By  1974,  Obermayer,  et  al  [67]  reported  that  an  economically  acceptable 
means  of  objectively  measuring  behavioral  skills  in  the  team  training  setting 
was  an  elusive  goal.  This  conclusion  was  echoed  by  the  Defense  Science  Board's 
1975  review  [74]  of  research  and  development  programs  in  military  training. 

Their  review  stated  that  the  team  performance  measurement  problem,  was  a 
"fundamental  stumbling  block  to  progress"  in  improving  team  training. 

While  a systematic  widely  applicable  methodology  for  team  training  evalua- 
tion is  not  yet  at  hand,  there  appears  to  be  a reasonable  consensus  as  to  what 
the  methodology  ought  to  accomplish  [8,  17,  50,  67].  The  literature  reviewed 
suggested  that  any  team  performance  measurement  system  must  address  the  following 

areas : 

1.  The  definition  of  team  performance  objectives  in  terms  of  specified, 
observable  outcomes  to  include  criteria  for  acceptance  and  conditions  of  per- 
formance. 

2.  The  definition  of  a metric  or  range  of  values  applicable  to  each 
specified  observable  event. 
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3.  The  detection,  measurement,  and  recording  of  the  value  of  an  observed 
event  at  each  occurrence. 

A.  An  evaluation  of  the  team  as  having  attained  or  not  attained  the 
objective — based  on  discrepancies  between  outcome  criteria  and  observed  event 
values . 

5.  The  feedback  of  team  performance  data  to  the  training  environment. 

The  remainder  of  this  chapter  will  discuss  current  developments  in  team 
training  evaluation  in  terms  of  the  first  four  areas  listed  above.'*"  In  terms 
of  the  two-dimensional  categorization  of  training  focus  and  task  situation, 

Table  6 shows  that  most  of  the  studies  reviewed  from  this  chapter  are  related  to 
the  evaluation  of  team  training  in  emergent  task  situations. 

The  Definition  of  Team  Performance  Objectives 

Much  has  been  written  on  procedures  for  deriving  training  and  performance 
objectives  [2,  82].  The  actual  statements  of  objectives  are  the  end  products 
of.  a series  of  steps  in  a systems  approach  to  training  development.  Under  this 
approach,  the  resulting  objectives  have  three  characteristi cs : they  must  describe 

behaviors  which  are  to  be  demonstrated  in  the  test  situation;  they  should  indicate 
the  conditions  under  which  these  behaviors  are  to  be  demonstrated;  and  they 
should  include  criteria  or  standards  of  performance.  The  absence  of  performance 
criteria,  usually  stated  in  terms  of  accuracy,  time,  casualties  taken,  tolerance 
limits,  etc.,  appear  to  be  a seriously  deficient  aspect  of  team  training  and 
performance  evaluation. 

2 

The  Army  Training  and  Evaluation  Program  (ARTEr)  , a body  of  training 
reference  literature  stressing  a mission-oriented,  performance-based  approach 


The  feedback  of  team  performance  data  v as  discussed  in  Chapter  II. 

2 

See  Chapter  IV  for  a more  detailed  description  of  ARTEP. 
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has  b(  an  under  development  since  1974.  Ir  197.-75,  the  9th  Infantry  Division 
and  thi  1st  Cavalry  Division  trained  and  tested  under  prototype  Army  Training  and 
Evaluation  x’rograms  (ARTEPs)  as  part  of  an  extensive  examination  of  tne  feasibility 
of  ARTEP  implementation.  One  of  the  primary  objectives  of  the  effort  was  to 
"assess  the  effectiveness  of  the  ARTEP  as  an  instrument  for  evaluating  unit 
proficiency"  [93].  It  was  concluded  that  the  standard^  of  performance  stated  for 
Army  Training  and  Evaluation  Program  missions  required  revision.  Army  T raining  and 
Evaluation  Program  standards  were  inaccurate  in  some  instances  and  too  general  and 
vague  in  many  others,  requiring  individual  rater  judgment  to  a greater  extent 
than  necessary.  Army  Training  and  Evaluation  Program  standards,  while  ostensibly 
stated  in  specific  performance  terms,  contained  many  indefinite  terms  such  as 
"on  time,"  "excessive,"  "sufficient,"  "promptly,"  "proper,"  etc.  Interpretation 
of  such  terms  is  bound  to  vary  widely  without  some  guidance.  One  major  difficulty 
relates  to  the  use.  of  multiple  standards  and  the  lack  of  guidance  as  to  the 
relative  value  of  each  component.  Such  heavy  reliance  upon  judgments  of  individual 
evaluators  can  lead  to  diminished  Army  Training  and  Evaluation  Program  validity 
and  reliability. 

Obermayer  [67]  described  a highly  sophisticated,  semi-automated  system  for 
evaluating  air  crew  performance.  He  stated  that  in  some  areas,  such  as  air 
combat  maneuvering,  combat  readiness  determination  is  purely  subjective.  Chesler 
[17]  asserted  that  standards  for  many  combat  situations  simulated  or  real 
are  often  matters  of  widely  differing  opinion.  Larson  [57]  described  a possible 
way  to  overcome  these  differences.  In  his  survey  of  team  performance  effective- 
usss  measures  he  suggested  the  use  of  the  Delphi  techuique-a  method  for  systematically 
extracting  expert  opinion— to  elicit  subjective  indicators  of  team  performance 
from  senior  Marine  Corps  personnel.  These  performance  indicators  could  then  be 
used  in  field  exercises  by  less  experienced  evaluators. 
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Definition  of  Observable  Event  Metrics 

The  critical  concern  in  proficiency  evaluation  is  whether  or  not  a team 
has  gotten  the  job  done.  It  is  the  objective  of  evaluation  indices  to  measure 
data  that  indicate  whether  the  defined  mission  has  been  & .ccessfully  performed. 
Precise  specifications  of  events  to  be  assessed  could  be  advanced  with  the 
implementation  of  the  Instructional  System  Development  (ISD)  approach  which 
subsumes  the  use  of  job /task  analysis  procedures.  However,  much  performance 
evaluation  in  training  situations  today  is  still  highly  subjective  [17]. 

Upon  examination,  this  problem  inevitably  boils  down  to  a lack  of  criterion 
performance  variables  which  are  objective,  recordable,  discriminatory,  and — most 
important — acceptable  to  a consensus  of  persons  familiar  with  the  tasks  and 
skills  of  concern.  This  is  the  criterion  dilemma,  which  is  especially  severe 
for  team  training.  Acceptable  indices  of  good  and  poor  performance  for  military 
operations  are  difficult  to  define  [17].  Nonetheless,  techniques  for  the 
systematic  identification  of  measurable,  related  events  have  been  developed. 

[8,  17,  67] 

Adoption  of  such  methods  may  well  provide  the  kind  of  comprehensive,  detailed 
information  necessary  to  the  development  of  valid  performance  criteria,  but 
which  at  present  is  generally  not  available  to  training  and  evaluation  personnel. 
The  Tactical  Advanced  Combat  Direction  Electronic  Warfare  (TAODEW)  System  appears 
well  thought  out  and  could  readily  serve  as  a model  approach  for  other  team 
performance  evaluation  efforts. 

The  basic  events  in  the  TACDEW  evaluation  approach  [17]  are  the  "performance 
variable"  and  the  "situational  variable." 


A performance  variable  is  a measure  of  the  accuracy  or  timeliness 
of  a trainee  response.  A situational  variable  is  a measure  of  some 
aspect  of  the  embedding  environment  for  the  trainee’s  response. 

Performance  variables  derive  from  two  types  of  data  elements:  actual 

trainee  response,  and  the  response  considered  correct.  The  performance 
variable  is  the  quantitative  difference  between  or  qualitative  compari- 
son of  the  two.  For  example,  reported  bearing  (trainee  response  data 
element)  ter  a target  is  320  degrees;  the  true  target  bearing  (correct 
response  data  element)  is  300  degrees;  the  performance  variable,  error 
in  reported  bearing,  is  20  degrees.  This  performance  variable  is 
a measure  of  accuracy  of  response  [17]. 

Performance  variables  relating  to  timeliness  of  trainee  response  are 
derived  from  the  actual  latency  of  the  trainee’s  response  and  a reference  or 
ideal  response  latency. 

A set  of  performance  (and  situational)  variables  is  then  related  to  an  event. 
Once  the  event  occurs,  preliminary  lists  of  performance  variables  are  generated 
which  identify  any  and  all  variables  that  show  promise  of  serving  as  measures  of 
training  objective  attainment.  The  developers  of  the  Tactical  Advanced  Combat 
Electronic  Warfare  (TACDEW)  system  anticipate  that  their  approach  will  help 
clarify  the  relationship  among  the  various  measurements  and  their  relevance  as 
criteria  for  making  adequate  judgments  regarding  team  training  outcomes. 

Using  similar,  but  perhaps  less  clearly  articulated  methods,  training 
evaluators  in  other  Services  are  attempting  to  define  objective-related  events. 

p , in  Project  REALTRAIN,  an  Army-supported  approach  which  uses  realistic 
two-sided,  free-play  exercises,  a casualty  assessment  technique  was  developed  for 
the  basic  infantry  weapon,  the  M16A1  rifle.  The  development  of  the  casualty 
assessment  technique  for  the  M16A1  and  the  development  of  similar  techniques  for 
other  infantry  and  armor  weapons  was  considered  a breakthrough  necessary  in 
simulating  the  tactical  environment  [79,  84].  As  part  of  the  REALTRAIN  procedures 
a sequential  record  of  events  is  kept  during  each  engagement  at  the  controller's 


See  Chapter  IV  for  a further  discussion  of  REALTRAIN. 
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station.  This  record  includes  each  casualty,  the  time  at  which  it  occurred,  and 
the  weapon  used. 

Statistical  analysis  can  help  to  reduce  the  number  of  possible  variables. 
Some  variables  may  correlate  highly  with  others,  and  can  be  eliminated.  Other 
variables  may  show  very  little  spread  in  range  of  scores,  so  that  they  do  not 
differentiate  among  trainees.  Project  NORM  (Normative  Operations  Reporting 
Method) , was  an  effort  by  the  System  Development  Corporation  to  identiiy  valid 
performance  variables  for  SAGE  [17]  team  evaluation,  ani  to  determine  how 
situational  variables  affected  team  performance  [76].  A basic  concept  of 
Project  NORM  was  that  situational  variables  were  measures  of  exercise  difficulty. 
An  exploratory  pilot  study  provided  data  for  41  variables — 24  performance  and 
17  situational  variables.  Correlational,  linear  multi-variate  regression,  and 
factor  analysis  techniques  were  used  to  reduce  the  original  number  of  41 
variables  to  13.  [96]  There  were  six  situational  variables  (related  to  the  task 
at  hand)  which  were  potential  contributors  to  performance.  In  addition,  seven 
performance  variables  (e.g.,  detection  latency)  were  considered  as  relevant  for 
evaluating  system  performance.  Whereas  the  variables  selected  appear  to  be 
useful  primarily  the  study  content,  the  analytic  methodology  employed  to  derive 
these  variables  is  applicable  to  emergent  task  situations,  in  general. 

Biel  [8]  described  another  SAGE  (Simulated  xiir-Ground  Environment)  study 
in  which  all  measures  used  to  evaluate  team  performance  in  a simulated  environment 
were  collected  during  a series  of  specially  prepared  simulated  exercises. 
Correlations  among  team  performance  variables  with  measures  of  overall  system 
performance  in  detection,  identification,  tracking  , interceptor  commitment, 
and  interceptor  guidance  were  computed  and  the  resulting  matrix  factoreu.  The 
results  of  this  study  recommended  the  deletion  of  those  measures  which  did  not 


have  a significant  relationship  to  overall-all  system  performance.  It  also 
recommended  that  a small  number  of  systematically  obtained  observations  be  used 
to  make  judgments  about  team  performance. 

In  Army  tank  crew  training  a large  number  of  performance  objectives  havfe 
been  developed  for  tank  gunnery  training  and  proficiency  assessment.  Project 
IDOC  (Identification  of  Test  Doctrine  for  Cost-Effective  Qualification  of  Tank 
Crews)  is  concerned  with  the  identification  of  a reduced  set  of  tasks  on  which  to 
base  the  development  of  cost-effective  performance  measures  that  are  applicable 
to  the  entire  set  of  objectives.  Accordingly,  a joint  HumRRO/American  Institutes 
for  Research  effort  was  undertaken  to  determine  the  component  team  behaviors  for 
each  objective,  and  the  commonality  of  behaviors  across  objectives.  Plans  are 
currently  being  formulated  to  analyze  data  obtained  so  far  of  generalities 
across  tank  gunnery  tasks,  psychometric  problems,  and  practical  constraings  on 
performance  assessment. 

Detection,  Measurement,  and  Recording  of  Observable  Events 

A problem  of  some  concern  is  the  unreliability  and  inaccuracy  of  trainers 
and  observers  in  evaluating  team  performance  in  the  field.  Since  there  is  no 
fixed  doctrine  of  what  to  observe  and  how  to  evaluate  it  for  various  team  acti- 
vities, the  tendency  is  for  inter-rater  reliability  to  be  quite  low. 

Larson's  1971  review  [57]  of  the  team  performance  measurement  literature^ 
concurs  with  this  assessment.  He  stated  that  there  was  a shortage  of  objective, 
quantitative  methods  available  for  application  in  field  exercise  environments 
caused  by  an  inherent  difficulty  in  the  measurement  of  complex  human  performance 
in  a field  environment.  In  REALTRAIN  field  exercises,  detection  and  measurement 
of  observable  events  (casualty  assessment)  is  performed  entirely  by  human 


Interview  with  Army  Research  Institute  staff. 
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controllers  [76].  The  light  anti-tank  weapon  (LAW)  and  90mm  recoiless  rifle  (RGLR) 
are  equipped  with  range-calibrated  sighting  plates  which  a controller  looks 
through  during  a simulated  engagement  to  determine  hits.  Casualties  with  the 
anti-tank  weapon  (TOW)  are  determined  by  a controller  sighting  through  an 
accessory  telescope  mounted  on  the  outside  of  the  weapon.  In  a recent  field  tryout 
of  the  REALTRAIN  method  for  combined  arms  tactical  training,  it  was  reported  that 
the  data  collected  were  limited  due  to  the  methodological  problems  of  quantitatively 
measuring  team  performance  [79]. 

Some  team  training  evaluation  efforts  attempted  to  circumvent  the  "unreliable" 
observer  problem  by  automating  detection  and  measurement  as  much  as  possible. 

This  is  particularly  true  in  areas  where  sophisticated  electronic  and  computing 
technology  are  employed  in  mission  fulfillment [4 , 8,  17,  67].  The  Army 
is  presently  investigating  the  use  of  laser  technology  for  this  purpose.  Project 
MILES  (Multiple  Integrated  Laser  Evaluation  System)  involves  the  integration 
of  lasers  with  a variety  of  weapons  types.  "Firing"  of  a weapon  emits  a weak 
laser  beam  which,  if  aim  is  accurate,  activates  a recording  device  on  the  target. 
The  MILES  (Multiple  Integrated  Laser  Evaluation  System)  is  still  experimental 
and  has  not  been  subjected  to  field  trials,  yet  it  has  good  potential  for  solving 
the  problem  of  unreliable  human  measurement.  Even  though  MILES  was  developed 
for  use  in  multi-individual  training  settings,  the  concept  could  be  applicable 
to  the  evaluation  of  crew-served  weapons  systems  training. 

Where  such  technological  solutions  are  not  to  be  used,  other  techniques  for 
minimizing  rater  effect  are  being  explored.  To  support  the  development  of 
systematic  methods  of  criterion-referenced  performance  assessment,  the  Army 
Research  Institute  through  project  METTEST  (Methodological  Issues  in  Criterion 
Referenced  Testing)  is  funding  several  investigations  into  the  applicability 
of  psychometric  methods  to  the  requiremer.ts  of  combat  team  performance  measure- 
ment. One  such  effort  addresses  the  question  of  which  tasks,  given  limited  time 
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and  resources,  should  be  assessed  via  high  fidelity  operational  simulations,  which 
may  be  measured  "synthetically"  (via  paper  and  pencil  instruments) , and  which 
may  be  assessed  by  inference  from  performance  on  related,  more  inclusive  tasks. 

Evaluation  of  Objective  Attainment 

Attempts  to  evaluate  team  performance  during  field  performance  tests  have 
varied  considerably.  In  some  cases,  it  is  assumed  that  if  a given  team  completes 
an  exercise  successfully,  the  team  received  training  in  relevant  vehaviors. 

The  most  common  means  of  evaluating  performance  is  overall  subjective 
evaluation.  Where  detailed  checklists  are  used,  each  individual  team  member 
receives  a numerically  weighted  score.  The  team  score,  then,  is  arrived  at 
through  some  weighted  combination  of  individual  scores  generally  a simple  sum. 

The  ability  of  human  evaluators  to  effectively  judge  the  success  of  teams 
in  complex  field  exercises  is  generally  considered  unsatisfactory.  In  the  Army 
Training  and  Evaluation  Program  (ARTEP)  validation  report  [93]  cited  earlier,  it 
was  concluded  that  the  ARTEP  in  its  present  form  is  not  a standardized  test 
instrument.  There  is  no  reason  to  expect  that  different  teams  would  be  evaluated 
under  the  same  conditions  when  using  Army  Training  and  Evaluation  Program 
guidance  and  standards.  The  standards  are  too  subjective  and  evaluator  performance 
is  too  erratic.  There  is  no  explanation  of  how  to  relate  task  performance  to 
overall  mission  performance  or  how  to  adjust  standards  to  account  for  varying 
test  conditions  [93]. 

The  Modern  Army  Selected  System  Test  Evaluation  and  Review  (MASSTER)  attempts 
to  evaluate  weapons  systems  training  by  employing  a combination  of  checklists, 
associated  logic  trees,  and  expert  judges  in  the  field  environment.  The  checklist 
logic  tree  evaluation  uses  field  observers  to  check  off  the  team  s performance 
in  specified  situations.  The  team  passes  or  fails  progressively  larger  tests 


as  a result  of  their  performance  on  previous  examinations.  MASSTER  also  assigns 
officers  to  subjectively  evaluate  large  teams  during  an  exercise  [57], 

Currently  experienced  difficulties  in  making  reliable  and  accurate  judgments 
about  team  performance  is  in  large  measure  due  to  the  multiplicity  of  potentially 
influential  factors.  Chesler  [17]  stated: 

. . . single  performance  measures  are  inadequate  for  making  overall 
evaluations  of  system  effectiveness,  presumably  because  multiple 
factors  are  involved  in  the  determ-* ip^'.on  of  mission  success  or  failure. 
Combining  measures  into  overall  indices,  has,  so  far,  not  seemed  to  help 
much,  probably  because  the  relationship  between  them  is  not  clearly 
understood.  Thus,  lumping  them  together  does  not  necessarily  improve 
the  quality  of  system  evaluation. 

Recognition  of  this  complexity  has  led  to  a number  of  investigations  of 
analytic  or  modeling  approaches  to  the  evaluation  problem.  A study  within  the 
Army  Research  Institute's  METTEST  (Methodological  Issues  in  Criterion  Referenced 
Testing)  project  is  concerned  with  the  applicability  of  traditional  measurement 
models  to  the  need  for  team  performance  indices.  Measures  are  needed  that  give 
valid  representation  to  the  many  team-related  and  environmental  factors  which 
influence  mission  attainment.  Preliminary  work  has  suggested  that  the  pre- 
dominantly linear  additive  models  of  psychometrics  are  not  applicable  in  this 
context  that  non-linear  (e.g.,  exponential)  and  "multiple-cutoff "-.models  may 
be  more  appropriate.  In  "multiple-cutoff-models,"  team  activities  are 
structured  in  a manner  related  to  the  sequence  in  which  they  would  occur  in  the 
field,  and  each  is  evaluated  on  a GO/NO-GO  basis.  Within  this  approach  "compen- 
satory" and  "non-compensatory"  activities  can  be  identified.  It  is  non- 
compensatory if  one  particular  component  will  abort  the  attainment  of  the  mission 
if  it  is  not  performed  properly.  If  some  other  component  can  successfully  pick 
up  the  slack  left  by  the  failed  component,  then  it  is  compensatory. 


The  IPAM  model  (Unit  Performance  Assessment  Model)  developed  at  the  Army 
Research  Institute  employs  casualty  data  as  a measure  of  both  the  achievements 
of  a team  (casualties  inflicted  on  an  opposing  force)  and  the  cost  to  the  team 
(casualties  sustained)  during  its  efforts  to  fulfill  a mission.  Measurement 
of  a team's  accomplishments  is  obtained  by  inputting  to  the  model  data  from 
REALTRAIN  field  exercises  and  deriving  a single  value  reflecting  both  cost  and 
achievement.  A criterion  value  for  assessing  the  empirical  result  is  obtained  by 
having  the  unit  commander  provide  estimates  of  these  values  that  in  his  judgment 
are  indicative  of  a successfully  accomplished  mission.  The  difference  between 
empirics’  output  and  the  commander's  "expected"  output  (on  a scale  from  -1  to  +1) 
is  viewed  as  an  index  of  the  team's  success  in  attaining  its  goal.  The  UPAM  model 
has  been  applied  to  both  infantry  and  armor  units.  However,  in  its  present  state 
of  development,  the  model  does  not  provide  evaluative  information  concerning  team 
activities  which  contribute  to  or  detract  from  mission  accomplishment. 

The  most  detailed  and  sophisticated  analytic  model  found  in  this  survey 
is  the  CARMONETTE  ground  close  combat  computer  simulation  [20].  Its  parameters 
include  detailed  descriptions  of  team  size,  team  type,  firepower,  vulnerability, 
mobility  and  sensing  capabilities.  Data  on  terrain  ranges  from  elevation  and 
height  of  vegetation  to  traf ficability.  Additionally,  a set  of  "orders"  can 
be  transmitted  to  each  team  of  opposing  forces  (i.e.,  a mist  •*  .)  . Further,  each 

team  can  acquire  information  and  relay  it  to  equivalent  teams  or  to  its  command 
HQ.  While  the  output  from  CARMONETTE  provides  only  descriptive  data,  it  could, 
if  provided  with  field  data,  provide  valuable  clues  concerning  the  effect  of 
performance  and  situational  variables  on  mission  outcomes. 


45 


» ■ - mi  pm  - . ..  m 


Surrnia  ry 

There  is  still  a major  unsolved  problem  in  the  development  of  evaluation 
methods  and  tools  for  team  training — namely,  there  is  a need  for  reliable, 
objective-based,  team  performance  measures.  The  literature  reviewed  indicated 
the  difficulties  of  doing  research  in  this  area  in  spite  of  its  acknowledged 
importance  to  the  improvement  of  team  training.  The  current  support  within  the 
Army  Research  Institute  for  R&D  in  this  area  demonstrates  their  realization  of  its 
criticality.  Most  of  this  research  is  still  in  progress.  Findings  gleaned  from 
the  studies  surveyed  suggested  the  following: 

• A systems  approach  to  team  training  development  should 
alleviate  some  of  the  problems  associated  with  the  establishment 
of  criteria/standards. 

• Much  performance  evaluation  in  team  training  today  is  highly 
subjective  and  unreliable. 

• Lacking  an  adequate  team  performance  measurement  system,  alternative 
team  training  programs  and  team  proficiency  cannot  be  properly 
evaluated. 

• Automating  the  monitoring  and/or  data  recording  process  is  needed 
in  order  to  solve  the  problem  of  unreliable  human  observers. 

Recent  innovations  in  team  training  methodology  discussed  in  Chapter  II 
promise  significant  improvements  in  operational  performance.  A strong  emphasis 
on  R&D  in  the  area  of  team  performance  measurement  is  required  in  order  that  this 
improvement  be  detected  and  operationally  realized. 


Chapter  IV:  CURRENT  MILITARY  TEAM  TRAINING  APPROACHES 


Examples  of  team  training  techniques  currently  in  use  in  the  Services  are 
presented  in  this  section.  It  is  not  intended  to  be  an  exhaustive  presentation 
of  such  training  methods.  Rather,  the  examples  were  chosen  to  represent  the 
variety  of  methods  now  employed  to  conduct  team  training.  The  training  devices  and 
systems  discussed  in  this  chapter  are  categorized  in  Table  7.  They  were  all 
classified  as  pertaining  to  emergent  task  situations  because  they  stress  the 
realistic  requirements  of  operational  environments. 

ARMY  TEAM  TRAINING 

Army  team  training  is  predominantly  a unit  training  function  with  no  formal 
institutional  training  requirements.  The  great  majority  of  personnel  who 
complete  Military  Occupational  Specialty  (MOS)  training  at  Army  schools  and 
training  centers  are  assigned  to  units  as  replacements.  They  receive  team  training 
in  the  units  in  the  form  of  individual  on-the-job  training,  augmentation,  cross- 
training, and  collective  (team)  training.  Most  training  is  performed  on  opera- 
tional equipment  with  the  objectives  of  maximizing  proficiency  or  readiness  [41,  88]. 

In  the  Army,  Advanced  Flight  Training  is  considered  team  training.  Graduates 
of  Undergraduate  Pilot  Training  receive  supplementary  training  in  the  specific 
aircraft  they  will  fly  on  operational  missions.  The  supplementary  training  is 
provided  by  the  operational  unit.  The  new  pilot  is  integrated  into  the  opera- 
tional unit  where  he  undergoes  transitional  training  as  part  of  normal  unit 
training  [74]. 
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EMERGENT  ESTABLISHED 


Much  of  the  team  training  conducted  in  the  Army  is  based  on  information 
contained  in  Army  Training  and  Evaluation  Program  (ARTEP)  publications.  That 
is,  ARTEPs  "drive"  the  Army's  team  training  system.  ARTEPs  are  developed  to 
assist  training  managers  and  trainers  in  the  preparation,  conduct  and  evaluation 
of  team  training.1  They  consis-  of  a series  of  training  and  evaluation  outlines 


TRAINING  SITUATION 

team  MULTI-INDIVIDUAL 


CATTS  (Combined  Arms  Tactical 
Training  Simulator)  [74] 

WST  (Device  2F87,  Weapons  System 
Trainer)  [41] 

Device  14A2,  Surface  Ship  Anti- 
submarine Warf are(ASW) , Early 
Attack  Weapons  Systems  Team 
Trainer  [41] 

Device  14A6,  ASW  Coordinated  Tactics 
Trainer  [41] 

Device  21A37/4,  Submarine  Fleet 
Ballistic  Missile  (FBM)  Training 
Facility  [41 ] 

TACDEW  (Tactical  Advanced  Combat 
Direction  and  Electronic  Warfare 
System)  [41 ,83] 

TESE  (Tactical  Exercise  Simulator 
and  Evaluator)  [75] 

FIST  (Functional  Integrated  Systems 
Trainer)  [74] 

C-5  MissionFlight  Simulator 

C-141  Flight  Simulator 

C-130  Flight  Simulator 

WST  (B—  52  Weapons  Systems  Trainer) 

SCOPES  (Squad  Combat  Operations 
Exercise,  Simulation)  [85,87] 

REALTRATN  [84] 

MILES  (Multiple  Integrated  Laser 
Engagement  System)  [84] 

TWAES  (Tactical  Warfare  Analysis 
and  EvaluationSystem)  [74] 

Table  7.  Classification  of  Training  Contexts 


^See  Chapter  III  for  a discussion  of  ARTEP  as  an  evaluation  mechanism. 
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which  specify  for  a particular  element  of  a battalion/task  force,  or  separate 
comoany,  the  following  information: 

a.  The  unit  (i.e.,  crew,  squad,  platoon,  company/team  or  batallion/ 
task  force)  for  which  the  outline  is  applicable. 

b.  The  mission  to  be  performed. 

c.  The  general  conditions  (situation)  under  which  the  mission  will 
be  performed. 

d.  The  primary  training  and  evaluation  standards  upon  which  the  uni; 
will  be  evaluated  as  either  satisfactory  or  unsatisfactory 

(Go /No  Go). 

e.  Th°  collective  training  objectives  applicable  in  the  performance 
of  the  mission;  guidance  for  estimating  support  requirements 
necessar  to  conduct  formal  evaluations,  and  tips  to  trainers  and 
evaluators  [85]. 

Inasmuch  as  the  Army  Training  and  Evaluation  Programs  (ARTEPs)  outline  the 
basic  missions  that  teams  should  be  able  to  perform  to  be  combat  ready,  they  are  a 
valuable  source  of  information  in  preparing  the  scenarios  for  training  strategies/ 
exercises  such  as  SCOPES  (Squad  Combat  Operations  Exercise,  Simulation),  MILES 
(Multiple  Integrated  Laser  Engagement  System) , REALTRAIN  and  CATTS  (Combined  Arms 
Tactical  Training  Simulator)  which  are  discussed  below. 

1.  SCOPES  (Squad  Combat  Operations  Exercise,  Simulation) 

SCOPES  was  originally  designed  to  teach  movement  techniques  to  rifle  teams 
and  squads.  It  has  been  successfully  used  to  train  and/or  evaluate  soldiers 
or  small  teams  on  a wide  range  of  battlefield  skills,  tasks,  or  missions. 

Briefly,  SCOPES  works  in  the  following  way.  Trainers  first  develop  a 
tactical  scenario  based  on  specified  training  objectives.  Six-power  telescopes 
mounted  on  each  rifleman's  M16A1  rifle  are  used  to  identify  numbers  worn  on  the 
helmets  of  all  participants  in  opposing  forces.  When  a number  of  an  "enemy" 
soldier  is  identified,  the  soldier  making  the  identification  fires  a blank  round 
and  calls  out  the  number  he  sees.  Controllers  in  the  vicinity  in  turn  pass  the 
number  over  a radio  net  to  controllers  operating  with  the  opposing  force.  If 


the  number  is  correct,  the  controller  immediately  "kills"  the  man  wearing  the 
number  seen. 

i I 

In  this  technique,  "battles"  may  be  conducted  with  real  outcomes.  Depending 
on  the  tactics  selected  and  the  individual  techniques  jused,  it  is  possib1e  to 

objectively  determine  "winners"  and  "losers."  I 

i 

After  each  engagement,  troops  on  both  sides  are  bf ought  together  for  an 
after-action  review  to  discuss  errors  made  and  lessons  learned.  In  this  manner, 
the  correct  techniques  to  avoid  being  "killed"  can  be  brought  out,  learned,  and 
practiced  in  subsequent  engagements  [85,  87]. 

1 

2.  REALTRAIN 

The  REALTRAIN  program  permits  the  conduct  of  larger' scale  exercises  using 
opposing  forces  than  does  SCOPES  (Squad  Combat  Operations  Exercise,  Simulation). 

To  simulate  the  effects  cf  other  weapons  systems,  optical  (devices  including 

i 

telescopes  and  plastic  sighting  plates  are  mounted  or  used  with  light  anti-tank 

weapons  (LAWs);  lOfmra  RCLR  (recoiless  rifles);  anti-tank  weapons  (TOWs);  tank 

I 

man  guns;  DRAGON  (medium  anti-tank  weapon);  and  90mm  RCLR. ! These  devices  are 
aligned  with  the  weapon's  sight  picture  thereby  allowing  controllers  ‘o  see  the 

same  sight  picture  as  the  gunners.  This  permits  them  to  verify  a c s aim 

! 

during  a target  engagement.  Gunners  "shoot"  at  target  . announcing  the  identi- 
fier tion  numbers  worn  by  the  soldiers  alLgned  in  their  sights.  Each  optical  device 
is  correlated  with  the  maximum  effective  range  of  the  particular  weapons  system 

with  which  it  is  used  [84].'*'  j 

! 

Field  tests  conducted  using  SCOPES  and  REALTRAIN  exeircises  have  proven  that 
engagement  simulation  techniques  work  for  training  multi-individual  groups.  Both 
soldiers  and  units  reach  higher  levels  of  tactical  proficiency  more  rapidly,  and 

soldiers  are  motivated  to  train  because  the  tactical  exercises  pr^''Tit  a real 

I 

challenge.  j 

j 

— 

1A  more  complete  description  of  REALTRAIN  can  be  found  in  Army  Training  Circular 
71-5  [84]. 
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3.  MILES  (Multiple  Integrated  Laser  Engagement  System) 

MILES  is  a group  of  experimental  training  devices  which  uses  an  eye-safe 
laser  beam  to  simulate  the  effects  of  weapons  expected  on  the  modern  battlefield. 
The  system  will  permit  conduct  of  day  and  night  tactical  exercises  and  will 
improve  the  integration  of  gunnery  techniques  into  tactical  training.  Some  of 
these  subsystems  are: 

• Vehicle  Engagement  System  (VES)  . The  VES  will  be  used  by  armor  units 
and  anti-armor  weapons  such  as  TOW  (tube-launched  optically  tracked 
wire  guided  missile) , DRAGON  (medium  anti-tank  weapon) , and  the  106mm 
recoiless  rifle. 

• Target  Engagement  Simulator  (TES) . The  TES  will  be  used  by  Infantry 
troops  to  provide  engagement  realism.  It  consists  of  a laser  trans- 
mitter and  a hit  indication  device  which  signals  "kills"  and  "near 
misses . " 

• Machine  Gun  Laser  (MGL) . The  MGL  simulates  machine  guns  both  unmounted 
and  mounted  on  tanks. 

MILES  will  permit  conduct  of  tactical  exercises  involving  battalion  and  task 

force  size  teams  [84].  It  is  scheduled  for  testing  in  1978. 

I', 

The  MILES  procedure  (as  well  as  SCOPES — Squad  Combat  Operations  Exercise, 
Simulation — and  REALTRAIN)  focus  on  inc1 vidual  skills  training  in  emergent  task 
situations.  The  feasibility  of  using  these  techniques  for  training  team  skills 
needs  to  be  determined. 


4.  CATTS  (Combined  Arms  Tactical  Training  Simulator) 

The  purpose  of  the  CATTS  system  which  is  still  under  development  is  to 
provide  Battalion  Commanders  and  their  staffs  with  simulated  combat  situations 
operating  from  a ground  command  post. 

The  CATTS  system  has  several  features  desired  for  training.  It  allows 
realistic,  real-time  ground  combat  simulation;  realistic  mockups  of  command  post 
vehicles;  actual  field  radios  for  communication;  on-the-spot  command  decisions 
and  critiques;  extensive  automation  to  assist  controllers;  and  cost-effective 


software  and  hardware. 
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The  simulator  utilizes  a Xerox  Sigma  9 computer  to  meet  the  following 
training  objectives: 

a.  Identify  the  relationships  that  exist  among  various  elements  and 
with  the  enemy. 

b.  Identify  alternative  courses  of  tion,  appropriate  formation 
maneuvers  and  fire  support  ap,  ;ions. 

c.  Communicate  decisions  to  subordinates  using  tactical  orders 
so  that  decisions  can  be  effectively  interpreted. 

d.  Manipulate  and  monitor  the  variety  of  tactical  radio  networks 
available  at  the  battalion  level.  [74] 

I . 

NAVY  TEAM  TRAINING 

As  in  all  of  the  Services,  team  trainin'-  in  the  Navy  consists  of  training 
which  is  provided  to  organized  teams  for  the  performance  of  a specific  mission. 

The  Navy  discusses  team  training  in  terms  of  five  categories:  Pre-team 

Indoctrination  Training;  Subsystem  Team  Training;  System  Sub-team  Training; 

System  Level  Operational  Training;  and  Multi-unit  System  Operational  Training’ [74]. 

Pre-team  Indoctrination  Training  is  conducted  in  a team  context  with  emphasis 
on  increasing  the  skill  levels  of  the  individuals  who  will  later  be  assigned  to 
operational  teams. 

In  Subsystem  Team  Training  teams  vary  in  composition  under  different 
conditions  of  shipboard  readiness.  The  teams  are  divided  into  three  departments. 

Combat  System  Teams  (e.g.,  Search  Radar  Detection  and  Tracking);  Unit  Operations 
Teams  (e.g.,  Navigation);  and  Engineering  Systems  (e.g.,  Firefighting/Damage 
Control) . 

System  Sub-team  Training  involves  the  training  of  two  or  more  sub-system 

teams  and  generally  an  entire  ship. 

System  Level  Operational  Training  is  accomplished  by  at-sea-training 
because  existing  pierside  facilities  do  not  have  the  capability  of  handling  an 
entire  crew  of  a ship  in  all  facets  of  an  operation.  This  type  of  training  is  best 


described  as  a battle  problem  which  is  conducted  by  underway  training  units  and 
fleet  training  groups. 

Multi-unit  System  Operational  Training  is  in-port  exercises  utilizing 
shore-based  trainers  for  the  purpose  of  training  crews  prior  to  getting  underway 
for  the  exercise  operations  area.  Major  in-port  exercises  utilize  the  Tactical 
Advanced  Combat  Direction  and  Electronic  Warfare  (TACDEW)  Trainer  [74]. 

The  following  major  team  training  devices  used  in  the  Navy  are  described 
below:  Device  2F87  (Weapons  Systems  Trainer),  Device  14A2  (Surface  Ship  Anti- 

Submarine  Warfare  Early  Attack  Weapons  Systems  Trainer),  Device  14A6  (Anti- 
Submarine  Warfare  Coordinated  Tactics  Trainer),  Device  21A37/4  (Submarine  Fleet 
Ballistic  Missile  Training  Facility),  and  TACDEW  [41]. 

1.  Device  2F87,  Weapons.  Systems  Trainer  (WST) 

The  WST  duplicates  the  interior  arrangement  and  appearance  of  the  P-3C 
aircraft.  Five  trainee  stations  simulate  corresponding  stations  in  the  aircraft. 

The  purpose  of  WST  training  is  to  teach  team  coordination.  Students  are  organized 
into  teams  according  to  positions  for  which  they  are  being  trained.  The  trainees 
"fly"  simulated  anti-submarine  warfare  (ASW)  missions.  These  missions  are  graduated 
in  difficulty  from  very  simple  scenarios  early  in  training  to  more  complex 
exercises  toward  course  completion. 

2.  Device  14A2,  Surface  Ship  ASW  (Anti-Submarine  Warfare)  Early  Attack  Weapons 
System  Trainer 

Device  14A2  is  used  to  train  surface  ship  teams  in  the  proper  utilization 
of  operational  ASW  systems.  Training  emphasizes  the  procedural,  tactical  decision 
making,  and  team  coordination  activities  in  operating  and  employing  ASW  weapons 
systems.  The  device  provides  for  indoctrination  of  personnel  in  ASW  procedures 
and  evaluation  of  tactical  situations.  The  trainer  is  also  used  in  developing 
and  planning  advanced  Naval  undersea  defense  tactics. 
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The  trainer  occupies  over  3000  square  feet  of  floor  space  and  duplicates 
the  physical  configuration  of  major  operational  compartments  and  equipments  of 
surface  ship  Anti-Submarine  Warfare  (ASW)  attack  weapons.  It  simulates  their 
functional  operation  and  responses  such  as  target  detection,  fire  control  solution, 
and  weapon  launching  and  tracking. 

3.  Device  14A6,  ASW  Coordinated  Tactics  Trainer 

Device  14A6  is  designed  to  train  decision-making  personnel  in  the  tasks 
they  must  perform  when  engaged  in  coordinated  ASW  tactics.  Simultaneous  operation 
of  48  vehicles  of  various  types  and  a multiplicity  of  sensors  can  be  simulated. 
Communications  facilities  simulate  the  various  radio  channels  employed  opera- 
tionally to  coordinate  all  phases  of  an  ASW  mission  from  search  through  attack. 
Device  14A6  provides  a synthetic  environment  within  which  ASW  personnel  can 
practice  collecting  and  evaluating  ASW  information,  making  decisions,  and  imple- 
menting the  decisions  based  on  this  information.  The  device  is  not  intended  to 
train  equipment  operators;  therefore,  simulated  equipment  does  not  resemble  fleet 
equipment.  Functional  characteristics  of  the  simulated  equipment  are  similar 
to  fleet  equipment. 

Virtually  any  exercise  at  sea  which  requires  communication,  coordination, 
maneuvering,  and  decision  making  may  be  practiced  in  the  14A6  trainer  prior  to 
going  to  sea. 

4.  Device  21A37/4,  Submarine  FleetBallistic  Missile  (FBM)  Training  Facility 

Device  21A37/4  provides  training  in  offensive  and  defensive  tactics  for 

nuclear  attack  cent  teams.  Surface  or  subsurface  maneuvers  may  be  accomplished, 
and  training  may  be  given  independently  or  in  coordination  with  other  units. 
Instruction  of  senior  command  and  staff  officers  in  direction  and  coordination 
of  submarine  task  groups  with  surface  support  units  may  also  be  given. 


. ~"V 


A central  digital  computer  provides  problem  generation,  position  and  motion 
data  generation.  Up  to  41  different  vehicles  can  be  included  in  training  problems. 
A projection  system  in  the  attack  centers  permits  both  in-progress  monitoring  and 
post-fire  analysis  of  training  problems.  Attack  centers  can  be  operated 
independently  or  operation  can  be  coordinated  to  provide  submarine  versus 
submarine  training.  Fifteen  different  classifications  of  targets  are  currently 
available,  12  at  any  one  time. 

5.  Tactical  Advanced  Combat  Direction  and  Electronic  Warfare  System  (TACDEW)1 
TACDEW  is  a highly  sophisticated  computer-based  simulationfacility  having 
three  primary  missions:  individual  and  team  training,  tactics  evaluation,  and 

testing  of  operational  computer  programs  for  Navy  Tactical  Data  System  (NTDS) 
ships.  Training  is  conducted  in  23  Combat  Information  Center  (CIC)  mcckups 
typical  of  the  ships  on  which  trainees  serve  [41].  Team  and  multi-team  training 
are  accomplished.  The  purpose  of  TACDEW  training  is  not  so  much  to  establish 
team  interactive  skills  as  it  is  to  maintain  or  enhance  these  skills  in  simu- 
lated mission  contexts.  Training  typically  consists  of  exercising  a given 
team,  or  some  combination  of  teams,  in  a series  of  scenarios  of  graded  difficulty. 
The  scenarios  are  designed  to  model  tactical  situations  which  might  be 
encountered  in  an  operational  environment. 


A more  detailed  description  of  the  TACDEW  system  is  presented  in  the  TACDEW 
Information  Pamphlet  [83]. 
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MARINE  TEAM  TRAINING 


The  Marine  Corps  has  recently  initiated  two  tactical  exercise  projects 
whose  objectives  are  to  develop  and  integrate  computer-driven  systems  into 
ongoing  field  and  school  training  programs.  The  computer-driven  simulation 
which  is  the  heart  of  these  training  systems,  can  handle  a greater  number  of 
factors  with  greater  precision  and  finer  resolution  than  was  previously  possible 
in  any  manually  conducted  model  [75]. 

The  two  team  training  systems  associated  with  these  projects  are  briefly 
discussed  below:  TESE  (Tactical  Exercise  Simulator  and  Evaluator);  TAWES 

(Tactical  Warfare  Analysis  and  Evaluation  System) . 

1.  TESE  (Tactical  Exercise  Simulator  and  Evaluator) 

TESE  is  used  to  train  Marine  Corps  officers  in  combat  decision  making. 

The  project  seeks  to  define  procedures  for  war  gaming.  The  goal  is  to  get  both 
computer-based  individual  and  team  measures  during  amphibious  warfare  exercises, 
and  to  increase  the  number  of  trainees  who  can  be  processed. 

2.  TWAES  (Tactical  Warfare  Analysis  and  Evaluation  System) 

TWAES  is  a computer-assisted  system  to  control  tactical  field  training 
exercises.  The  system  offers  potential  improvements  in  maneuver  control  and 
simulation  of  indirect  fire  [74].  The  current  TWAES  research  effort  is  studying 
the  exact  role  that  the  post-exercise  session  will  play  in  the  total  tactical 
exercise.  The  question  being  addressed  is  whether  this  feedback  session  will  be 
a training  vehicle  to  further  extend  the  game's  learning  effectiveness  or  whether 
it  will  be  merely  a post-exercise  debriefing  during  which  administrative  informa- 
tion about  game  procedures  is  passed  on  [75]. 


AIR  FORCE  TEAM  TRAINING 


In  the  Air  Force,  there  is  very  little  formal  team  training  being  conducted. 
All  such  formal  courses  surveyed  were  used  in  training  strategic  missile  teams. 
Students  are  provided  training  in  the  responsibilities  and  necessary  technical 
procedures  of  the  appropriate  weapons  system,  but  their  proficiency  upon  gradua- 
tion from  team  training  courses  is  less  than  that  required  for  combat-readiness. 
Thus,  additional  upgrade  training  is  required  at  the  unit/base  prior  to  actual 
job  performance  [74]. 

Similarly,  there  are  few  team  training  devices.  One  device  uncovered  in  this 
survey  is  the  Functional  Integrated  Systems  Trainer  (FIST)  . The  purpose  of  FIST 
is  to  provide  a better  means  for  training  four  members  of  the  fire  control  team 
on  the  AC-130E  Gunship.  The  second  objective  of  FIST  is  to  refine  and  promote  the 
use  of  the  technology  for  developing  low-cost,  interlinked,  functional,  part-task 
trainers  [74]. 

Similarly,  most  team  training  devices  deal  with  the  training  of  aircraft 
flight  crew  members.  Representative  training  devices  are  available  for  the  B52, 

C5  C130  and  C141  aircraft  as  described  below. 

1.  B52  (G  & H models)  Weapons  System  Trainer  (WST) 

The  WST  is  presently  under  development.  It  will  permit  the  integration 
into  a single  team  training  device  four  individual  devices  presently  being  used. 

The  four  devices  are:  (a)  The  Mission  Trainer  used  for  training  pilots  and  co- 

pilots; (b)  Navigator  Trainer  used  for  training  radar  navigators  ar.d  navigators; 

(c)  Electronic  Warfare  Officer  Trainer  (EWO) ; and  (d)  Gunnery  Trainer.  The  WST 
when  developed  will  permit  simultaneous  team  training  of  the  entire  six-man 
B52  crew.-*- 

^Personal  communication  with  COL  Roy  L.  Ripley,  LTC  Walter  D.  Black  III,  Instruc- 
tional Systems  Branch,  Audiovisual  and  Instructional  Systems  Division,  US  Air  Force 
Washington,  DC. 
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2.  C5  Mission  Flight  Simulator 

This  team  training  device  consists  of  three  training  stations  which  permit 
integrated  team  training  pilots/co-pilots , navigator  and  flight  engineer. 

3.  Cl 30  Flight  Simulator  and  C141  Flight  Simulator 

Both  of  these  training  devices  are  similar  to  the  C5  Mission  Flight  Simulator 

except  that  training  is  not  provided  for  the  navigator  station.  However,  the 

C130  Flight  Simulator  is  scheduled  to  add  the  navigator  station  as  part  of  the 
2 

device  in  1981. 

4.  Functional  Integrated  Systems  Trainer  (FIST) 

The  purpose  of  FIST  is  to  provide  a better  means  for  training  four  members 
of  the  fire  control  team  on  the  AC130E  Gunship . The  second  objective  of  FIST  is 
to  refine  and  promote  the  use  of  the  technology  for  developing  low-cost,  inter- 
linked, functional,  part  task  trainers  [74]. 

SUMMARY 

The  emphasis  in  this  survey  has  been  on  formal  team  training.  Much  informal 
team  on-the-job-training  occurs  in  the  daily  operations  of  the  Services.  It  is 
extremely  difficult  to  delineate,  in  such  activities,  the  training  mode  from  the 
operational.  This  is  especially  true  in  areas  such  as  flight  training,  air  defense, 
etc. 


^Personal  communication  with  LTC  C.R.  Philbrick,  MAC/DOTF , Scott  AFB,  IL. 
2 Ibid. 


However,  various  methods  of  team  training  are  currently  underway  in  the 
Services.  These  include  the  following: 

• Large-scale  opposing  forces  tactical  exercises 

• Devices  which  simulate  effects  of  weapons 

• Computer  simulations  of  tactical  decision-making  problems 

• Simulations  of  mission-specific  hardware  and  software  (e.g.,  anti- 
submarine warfare  vehicles. 

The  above  methods  are  tailored  for  both  initial  team  training  and  maintenance 
of  already  acquired  team  skills.  They  vary  in  cost,  simulation  fidelity,  and 
generalizeability.  Although  these  devices  are  now  being  used,  there  has  been 
relatively  little  programmatic  R&D  performed  upon  them  considering  the  cost  of 


such  systems. 


Chapter  V:  RESEARCH  IMPLICATIONS 


A number  of  implications  for  research  in  team  training/evaluation  are 
presented  in  this  chapter.  These  implications  resulted  from  a comparison 
of  current  military  practice  with  the  findings  of  the  literature  review. 

In  addition,  HumRRO  staff  relied  or  their  experience  in  the  training  re- 
search area  to  identify  the  most  critical  areas  for  support.  The  specific 
questions  addressed  in  this  report  were  the  following: 

(1)  What  state-of-art  gaps  are  there  in  team  training  strategies  and 
evaluation  techniques? 

(2)  What  new  team  training  and  evaluation  strategies  appear  to  hold 
promise  for  applications  to  the  DoD  environment  and  warrant 
further  study? 

The  literature  reviewed  for  this  report  support  the  contention  of  the 
Defense  Science  Board  [74]  that  there  is  a great  need  for  R&D  in  team 
training,  considering  the  magnitude  of  such  training  in  the  Services. 

One  major  problem  to  overcome  is  the  tremendous  diversity  of  terms,  con- 
cepts and  operations  which  are  used.  A "team"  is  not  a simple  fixed  unit, 
it  is  arbitrary,  has  unclear  boundaries,  and  an  unstable  structure  and  com- 
position. A team  should  be  defined  on  the  basis  of  its  criterion  situation, 
not  on  the  basis  of  administrative  convenience.  [34]  Describing  team  train- 
ing in  the  Services,  the  Defense  Science  Board  [74]  resorted  to  an  awkward 
abbreviation,  CGTU,  for  Crew,  Group,  Team,  and  Unit  to  accommodate  users 
of  each  of  these  terms.  There  is  a need  for  more  standardization  and  sharing 

of  terms.  Efforts  should  be  supported  to  develop  a reasonable  taxonomy 
upon  which  to  build  research  programs  in  team  training. 
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A classification  scheme  was  introduced  in  Chapter  3 which  was  used  to 
categorize  the  studies  and  techniques  discussed  in  this  review  along  two 
dimensions.  The  training  focus  can  be  either  team  or  multi-individual 
depending  upon  the  skills  to  be  trained.  The  task  situation  can  be  classified 
as  established  or  emergent. 

Methods  are  needed  for  differentiating  individual,  multi-individual,  and 
team  skills.  Techniques  need  to  be  developed  which  can  be  used  to  identify 
individual  and  team  skill  training  requirements  as  part  of  job/task  analyses. 
Such  techniques  would  discriminate  between  isolated  and  interactive  behaviors 
and  will  provide  the  opportunity  for  establishing  criteria/standards  of  team 
performance  as  part  of  training  requirements  analyses.  Variations  and/or 
combinations  of  observational  interaction  analyses  and  other  unobtrusive 
measures  which  have  been  used  in  non-classroom  settings  should  be  studied 
for  their  possible  application.  A recent  compendium  [95]  of  such  methods 
provides  detailed  information  about  techniques  which  are  of  potential  value 
such  as: 

• Interaction  Process  Analysis  — Robert  F.  Bales 

• Sequential  Analysis  of  Verbal  Interaction  (SAVI)  — Anita  Simon 

Yvonne  Agazarian 

© CERLI  Verbal  Behavior  Classification  (CVC)  — Cooperative  Educational 

Research  Laboratory,  Inc. 

The  development  of  such  analytic  techniques  is  a critical  first  step  in 
any  research  program  which  seeks  to  use  realistic  team  performance  standards 
as  effectiveness  criteria. 

There  is  a considerable  need  for  measures  of  team  performance  to  apply 
both  during  and  following  training.  As  described  in  Chapter  HI  this 
problem  has  resisted  solution  for  the  past  20  years.  The  difficulties  of 
observing  team  member  interactions , the  great  costs  in  money  and  time  to 
conduct  studies  in  this  area,  and  the  unclear  relationships  between  task  or 
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training  variables  and  team  performance,  have  all  contributed  to  the  un- 
successful search  for  measures  of  team  proficiency.  The  production  of 
stand^rdizable,  relatively  invariant  test  conditions  for  evaluating  dynamic 
and  interactive  team  behavior  remains  one  of  the  major  problems  in  team 
evaluation. 

It  is  only  in  recent  years  that  considerable  research  support  has  been 
provided  to  arrive  at  measures  of  team  performance.  Critical  problems  to  over- 
come are : 

• The  criterion  problem — Without  agreed-upon  criteria  or  standards  of 
team  performance,  evaluation  of  training  or  team  proficiency  cannot 
occur. 

• The  measurement  problem — Stated  succinctly,  what  to  measure,  where, 
when,  and  how,  are  still  all  unanswered  questions  in  the  evaluation 
of  team  performance. 

Considering  the  current  lack  of  information  in  this  area,  the  potential 
heuristic  value  of  the  analytic  or  modeling  approach  is  worthy  of  continued 
support.  Of  course,  the  usefulness  of  any  model  depends  on  its  ability  to 
generate  information  and  ideas  which  lead  to  the  development  of  practical 
and  worthwhile  evaluation  tools  and  techniques. 

For  example,  the  CARMONETTE  simulation  outputs  predictions  of  team  per- 
formance based  on  hypothetical  values  for  a number  of  performance  and  situa- 
tional variables  pertinent  to  the  team  and  to  an  opposing  force  (e.g.,  size/ 
vulnerability,  firepower,  mobility,  and/terrain  features) . Empirical  values 
gathered  from  field  exercise  data  for  the  variables  could  then  be  input  to 
the  simulation  and  outcomes  compared.  Continued  systematic  iterations  of  the 
model  would  suggest  the  specific  variabxes  that  made  a difference.  If  validated 
in  further  field  exercises,  these  variables  could  guide  the  refinement  of  test 
situations  and  the  development  of  better  measurement  techniques. 
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It  is  probably  true  that  teams  operate  in  situations  which  have  a 
combination  of  both  emergent  and  established  characteristics. 

This  underlies  the  imnortance  of  performing  a thorough  analysis  of  the 
criterion  task  situation  prior  to  developing  team  training  programs.  An 
application  of  the  systems  approach  to  the  development  of  team  training 
should  permit  identification  of  the  interactions,  communication,  coordination, 
decision  making  and  other  activities  required  in  the  task  performance  of 
each  team  member.  The  information  gathered  with  the  appropriate  analytical 
tools  should  include  standards  and  relevant  conditions  of  performance.  This 
would  provide  a major  step  toward  obtaining  clearly  stated,  objective  criteria 
and  procedures  for  evaluating  team  performance. 

One  consistent  finding  of  this  literature  review  was  that  individual 

proficiency  has  been  shown  to  be  a major  determiner  of  team  performance.  This 

had  led  investigators  such  as  Hall  and  Rizzo  [41]  to  conclude  chat  more  emphasis 

should  be  placed  upon  individual  rather  than  team  training.  The  authors  of 

this  review  disagree.  Although  certain  laboratory  studies  showed  that  when  the 

tasks  required  only  individual  skills,  team  training  was  superfluous,  other 

investigations,  primarily  in  more  realistic,  "emergent"  situations  showed 

the  importance  of  team  training  When  "team  skills"  (e.g.,  interaction),  even 

though  poorly  defined,  were  i important  in  the  task  situation,  team  training 

was  more  effective  than  individual  training.  Further  research  into  this  area 

should  provide  information  which  can  impact  on  the  issue  of  where  and  when  to 
conduct  team  training.  For  example,  individuals  in  air  squadrons  are  trained 

within  a team  context.  They  are  then  individually  assigned  to  operational 

units.  If  team  skills  are  relevant  to  a given  task  situation  and  can  be 

learned  during  a limited  training  period  it  may  be  more  efficient  to  assign 

the  team  intact  rather  than  disband  it  following  training  [41]. 
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A need  exists  for  studies  which  explore  the  id sue  of  skill  training 

i 

sequencing.  After  individual  and  team  skill  requirements  for  a given  task 
situation  have  been  idenitified,  the  sequence  in  wfhich  component  skills  are 

I 

trained  can  be  systematically  varied  and  the  resulting  effects  compared. 

' 

For  example,  individual  skills  can  be  trained  prior  to  team  skills  training; 

training  of  team  skills  can  occur  first;  or  both  individual  and  team  skills 

{ 

can  be  integrateu  within  the  training  program  ancf  taught  concurrently.  The 
results  of  such  studies  which  can  be  performed  in  either  established  or 
emergent  task  situations  will  help  determine  whei  and  under  what  conditions 

team  training  can  be  administered  most  effectively. 

f 

Some  types  of  team  training  are  so  costly  that  one  cannot  conduct  them  in 
an  operational  environment.  Other  types  of  training  have  logistic  or  safety 
factors  which  preclude  operational,  full  fidelity  training  or  testing.  High 

i 

l 

fidelity  simulations  which  were  common  in  individual  training  for  many  years 

i 

has  been  shown  to  be  effective  for  team  training.  However,  such  simulation 
techniques,  though  costly,  provide  little  generality  beyond  a specific  training 
mission.  A substantial  research  and  development  effort  is  needed  to  determine 
the  degree  of  simulation  fidelity  required  for  effective  training  in  different 
situations.  Relatively  low-fidelity  simulation  techniques  have  been  effective 
and  economical  in  individual  training.  However,  the  review  of  the  literature 
indicated  that  high  fidelity  simulations  were  needed  for  effective  team 
training.  The  studies  surveyed  were  performed  in  established  laboratory 
settings.  The  search  should  continue  for  effective  low  fidelity  devices  or 
methods  to  apply  to  team  training  in  emergent  task  situations  because  of 
their  potential  cost  savings. 


The  second  question  addressed  in  this  review  involved  the  identification 
of  team  training/evaluation  techniques  which  warranted  further  study  and 
support.  One  of  the  most  promising  new  approaches  is  two-sided  engagement 
training.  Current  implementations  of  this  technique  were  described  in 
Chapter  IV  in  discussions  of  SCOPES,  REALTRAIN,  and  MILES.  The  two-sided 
engagement  simulation  provides  a low  cost,  high  fidelity  framework  for 
developing  individual  skills.  It  provides  a situation  in  which  there  is 
high  motivation  fostered  by  competition  in  a natural,  realistic  environment. 

This  approach,  which  can  be  classified  as  situational  or  contextual  training 
is  an  outgrowth  of  the  effective  "functional  context  approach  to  individual 
training  [78].  Up  to  now,  two-sided  engagement  training  has  been  used  to 
teach  individual  skills  within  a group  context.  Support  is  needed  to  study 
the  feasibility  of  applying  these  methods  to  train  team  skills.  There  is  a 
need  to  study  how  effective  the  techniques  are,  analyze  the  factors  which 
produce  favorable  training  outcomes  and  study  how  these  methods  can  be 
transferred  to  other  training  situations.  Thus  far,  post— exercise  debriefings 
have  been  used  to  discuss  individual  performances.  Team  performance  infor 
mation  needs  to  be  fed  back  to  trainees  in  a more  timely  and  effective  manner 
both  during  and  after  training. 

A systematic  investigation  of  feedback  effects  upon  team  training  is  needed. 
Support  is  required  to  do  research  on  the  following  issues: 

• Need  for  feedback  - What  changes  in  training  methods  or  content  result 
from  feedback?  Will  there  be  differences  if  remediation  is  contingent 
or  non-con t indent  upon  feedback? 

• Feedback  mode  - Methods  should  be  available  for  giving  both  trainees 
and  trainer  feedback  of  how  well  the  team  is  performing.  Determining 
what  are  the  appropriate  methods  for  giving  performance  feedback  to 
team  members  is  an  area  requiring  further  R&D  (e.g..  Compare  video 
recording  with  post-exer  :ise  debriefing  procedures  — Hypothesis: 

Clarity  and  objectivity  of  observation  will  provide  more  useful 
feedback).  The  use  of  television  and  video  recording  to  provide 


65 


relatively  responsive  performance  feedback  has  beeu  used  successfu  y 
in  a variety  of  contexts  (e.g.,  sports,  teacher  training).  Its  potent  a 
value  to  military  team  training  appears  great,  but  needs  considerable 
R&D  support  to  identify  its  most  appropriate  applications. 

• Delav  of  feedback  - A systematic  study  is  needed  to  determine  under 
what  conditions  is  provision  of  feedback  immediately  after  a training 
exercise,  superior,  equal  to,  or  inferior  when  compared  to  delayed 
presentations— 1,  2,  or  more  days  following  the  event. 

Another  issue  which  merits  study  concerns  the  effects  of  team  member 


familiarity  on  the  acquisition  of  team  skills.  Does  the  length  of  time  that 
a team  functions  together  affect  team  performance.  One  could  systematically 
vary  this  time  parameter,  re-allocate  team  members,  set  up  new  teams,  and 


then  study  transfer  effects. 

The  ability  to  evaluate  oneself  accurately  is  a skill  which  should  help  to 
produce  effective  team  performance.  Analysis  of  one's  own  errors,  as  well  as 
awareness  that  one  is  overburdened  and  needs  other  team  members'  help  are  two 
specific  components  of  this  ability  which  should  be  trained.  There  is  a need 
for  a program  supporting  the  development  and  evaluation  of  training  packages 
which  would  provide  generalizable  instruction  in  these  self-assessment  skills. 
One  could  compare  subsequent  team  performance  with  or  without  prior  self- 
assessment  training.  Greater  proficiency  would  be  expected  from  those  teams 

who  have  had  such  self-assessment  skill  training. 

Another  possibility  is  that  prior  training  of  team  members  to  assess  the 
capabilities  of  others  will  result  in  better  subsequent  team  performance. 

By  having  the  ability  to  assess  other  team  members'  strengths  and  weaknesses, 
it  is  hypothesized  that  the  individual  team  member  best  suited  to  take  over 
or  help  out  on  a given  activity  can  be  selected  adequately  and  thereby 
enhance  overall  team  performance. 
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It  may  be  more  efficient  to  do  research  on  a combination  of  some  of  the 
issues  discussed  above  within  the  same  study.  For  example,  groups  feedback 
or  skill  sequencing  issues  can  be  studied  in  environments  which  contain 
established  or  emergent  task  situations  and  which  vary  in  their  degree  of 
simulation  fidelity. 

The  research  and  development  implications  mentioned  in  the  paragraphs 
above  are  merely  the  foci  of  what  should  be  considered  programmatic  efforts 
in  the  team  training/evaluation  area.  Considering  the  importance  and  extent 
of  this  area,  the  current  state  of  knowledge  is  quite  meager.  Studies  and 
techniques  such  as  those  mentioned  above,  if  supported,  should  help  to 
ameliorate  this  condition. 
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